Bloat in the Emacs Windows package

Bloat in the Emacs Windows package

[Björn Lindqvist]

Hello Emacs developers,

I noticed that the Emacs build of Windows has grown a lot so I started
a thread in help-gnu-emacs called "Emacs 26.1 on Windows is HUGE" and
was referred to this list. So see that thread for background info. The
numbers for the Emacs 24.5.1 and 26.1 Windows files are:

    emacs-26.1-x86_64-no-deps.zip download size: 106M disk size: 410M
    emacs-26.1-x86_64.zip download size: 210M disk size: 725M
    emacs-24.5-bin-i686-mingw32.zip download size: 48M disk size: 167M

For comparison jdk-11.0.2 consumes 278M.

The emacs-26.1-x86_64.zip file is really big. It contains a lot of
files which I wonder why they are necessary. Some examples

    python2.7.exe
    gdbus.exe
    libgdk_pixbuf-2.0-0.dll
    include/jasper/
    include/GL
    include/gnutls AND include/openssl
    lib/systemd
    sqlite3200.dll
    lib/pkgconfig
    lib/cmake
    share/bash-completion
    share/vala
    ...

The emacs-26.1-x86_64-no-deps.zip installation is smaller, but still
double the size of the corresponding 24.5 installation. This seem to
be because all binaries now include debugging symbols. Some examples
of the size increases:

    addpm.exe 577 kB => 2 282 kB
    ctags.exe 956 kB => 3 245 kB
    emacs.exe 8 989 kB => 121 740 kB
    emacs-24.5.exe 8 989 kB => 121 740 kB (emacs-26.1.exe)

The emacs.exe growth is especially annoying because the file is
copied. This seems like poor packaging to me. Why not have an
emacs.bat file calling emacs-26.1.exe and immediately save 121M?

According to the thread on help-gnu-emacs Emacs binaries used to be
stripped of debugging symbols, but aren't anymore and that is what is
causing the size increase. I wonder if we can return that? Most
software ported to Windows, such as MinGW, strips debugging symbols so
it is customary. For most users they are useless because they don't
run emacs.exe in gdb.exe.

For people with limited disk space, metered internet or old hardware
the new, bigger Emacs is a nuisance. On my machine it increases Emacs
start time by a second (although I don't know if that is caused by the
debugging symbols or if it is something else). It is also
aesthetically displeasing -- hackers like minimalism and hate bloat.

And while on the subject of Windows packaging. How come there is no
MSI installer for Emacs? It shouldn't be to hard to put one together
and it would make Emacs a little easier to install for newbies.


-- 
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

On April 17, 2019 8:01:56 AM GMT+03:00, "Björn Lindqvist" <bjourne@gmail.com> wrote:
> The emacs-26.1-x86_64.zip file is really big. It contains a lot of
> files which I wonder why they are necessary. Some examples
> 
>     python2.7.exe
>     gdbus.exe
>     libgdk_pixbuf-2.0-0.dll
>     include/jasper/
>     include/GL
>     include/gnutls AND include/openssl
>     lib/systemd
>     sqlite3200.dll
>     lib/pkgconfig
>     lib/cmake
>     share/bash-completion
>     share/vala
>     ...

Some people want the binary zip to include all the optional features that Emacs on Windows can support.  This zip includes all of the dependencies for those optional features.  Arguably, some of the auxiliary files, like header files and import libraries, could be omitted, but determining which ones are required is a very non-trivial and time-consuming job, so I can understand why Phillip, who volunteered to produce the binary zips, didn't do that.  This "one cannot fit all" problem is why we also have the bare-minimum zip with only the dependencies that are absolutely required.

> The emacs-26.1-x86_64-no-deps.zip installation is smaller, but still
> double the size of the corresponding 24.5 installation. This seem to
> be because all binaries now include debugging symbols. Some examples
> of the size increases:
> 
>     addpm.exe 577 kB => 2 282 kB
>     ctags.exe 956 kB => 3 245 kB
>     emacs.exe 8 989 kB => 121 740 kB
>     emacs-24.5.exe 8 989 kB => 121 740 kB (emacs-26.1.exe)

Stripping emacs.exe produces a 29MB file for Emacs 26.2.

We could perhaps move the debug info to a separate file and distribute it in a separate zip archive, but whether Phillip can afford that is entirely up to him.  He does that on his own free time; the Emacs project doesn't produce any "official" binaries, on any system.

> The emacs.exe growth is especially annoying because the file is
> copied. This seems like poor packaging to me. Why not have an
> emacs.bat file calling emacs-26.1.exe and immediately save 121M?

What you see in the zip is what the stock Emacs build procedure produces (except that originally the two Emacs executables are hard links to the same file data; but zip format doesn't support hard links, AFAIK).  The reason for that is to allow installation of additional versions on the same filesystem tree.

We don't provide any shell scripts or batch files because the build on Posix systems doesn't.  Once again, it's hard to blame volunteers for using the build products as is, without adding any more work.

> According to the thread on help-gnu-emacs Emacs binaries used to be
> stripped of debugging symbols, but aren't anymore and that is what is
> causing the size increase. I wonder if we can return that? Most
> software ported to Windows, such as MinGW, strips debugging symbols so
> it is customary. For most users they are useless because they don't
> run emacs.exe in gdb.exe.

It is a great help to have debug info when problems are reported that cause crashes and cannot be easily reproduced.  But if Phillip can afford prodicing separate debug info file for Emacs, we could have the cake and eat it, too.    The other programs can be stripped, as far as Emacs is concerned (but they are much smaller, so the savings in disk space will be small).

> On my machine it increases Emacs
> start time by a second (although I don't know if that is caused by the
> debugging symbols or if it is something else). 

It cannot be due to debug info, because that is not read when the program loads.

> It is also
> aesthetically displeasing -- hackers like minimalism and hate bloat.

FWIW, I think you the first one to complain about this.

> And while on the subject of Windows packaging. How come there is no
> MSI installer for Emacs? It shouldn't be to hard to put one together
> and it would make Emacs a little easier to install for newbies.

What tools to use to produce the binary distribution is entirely up to the person who does that.  And of course MSI is not free software.

Re: Bloat in the Emacs Windows package

[Van L]

Eli Zaretskii <eliz@gnu.org> writes:

>> And while on the subject of Windows packaging. How come there is no
>> MSI installer for Emacs? It shouldn't be to hard to put one together
>> and it would make Emacs a little easier to install for newbies.
>
> What tools to use to produce the binary distribution is entirely up to
> the person who does that.  And of course MSI is not free software.

WSL should require less efforting in translation from GNU/Linux packaging.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
"Pilots are no longer needed, but rather computer scientists" - Donald Trump

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

On April 17, 2019 2:15:44 PM GMT+03:00, Van L <van@scratch.space> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> And while on the subject of Windows packaging. How come there is no
> >> MSI installer for Emacs? It shouldn't be to hard to put one
> together
> >> and it would make Emacs a little easier to install for newbies.
> >
> > What tools to use to produce the binary distribution is entirely up
> to
> > the person who does that.  And of course MSI is not free software.
> 
> WSL should require less efforting in translation from GNU/Linux
> packaging.

There's no GNU/Linux packages, there are only files installed by "make install".

Re: Bloat in the Emacs Windows package

[Stefan Monnier]

> Some people want the binary zip to include all the optional features that
> Emacs on Windows can support.  This zip includes all of the dependencies for
> those optional features.  Arguably, some of the auxiliary files, like header
> files and import libraries, could be omitted, but determining which ones are
> required is a very non-trivial and time-consuming job, so I can understand
> why Phillip, who volunteered to produce the binary zips, didn't do that.
> This "one cannot fit all" problem is why we also have the bare-minimum zip
> with only the dependencies that are absolutely required.

Indeed.  But I wonder about licensing issues of some of those binaries:
for those packages which are (L)GPL'd, do we distribute the
corresponding source somewhere, like the license requires us to do?

>>     addpm.exe 577 kB => 2 282 kB
>>     ctags.exe 956 kB => 3 245 kB
>>     emacs.exe 8 989 kB => 121 740 kB
>>     emacs-24.5.exe 8 989 kB => 121 740 kB (emacs-26.1.exe)
> Stripping emacs.exe produces a 29MB file for Emacs 26.2.

I wonder what caused the 9MB => 29MB jump between 24.5 and 26?

> We don't provide any shell scripts or batch files because the build on Posix
> systems doesn't.  Once again, it's hard to blame volunteers for using the
> build products as is, without adding any more work.

FWIW, I think saving a 100MB is worth the extra work.

(but, indeed, using hard-links is the better option when available:
I often just `rm src/emacs-[0-9]*` to get rid of old Emacs releases it'd
be much trickier to do if src/emacs were a script that runs
src/emacs-27.0.50.5).


        Stefan

Re: Bloat in the Emacs Windows package

[Stefan Monnier]

>> And while on the subject of Windows packaging. How come there is no
>> MSI installer for Emacs? It shouldn't be to hard to put one together
>> and it would make Emacs a little easier to install for newbies.
> What tools to use to produce the binary distribution is entirely up to the
> person who does that.  And of course MSI is not free software.

BTW, I seem to remember someone working on an "installer" for Emacs
on Windows.  Does anyone know what came out of that?


        Stefan

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 17 Apr 2019 08:39:52 -0400
> 
> FWIW, I think saving a 100MB is worth the extra work.

We cannot make that decision, only Phillip can.

Re: Bloat in the Emacs Windows package

[Björn Lindqvist]

Den ons 17 apr. 2019 kl 09:31 skrev Eli Zaretskii <eliz@gnu.org>:
>
> On April 17, 2019 8:01:56 AM GMT+03:00, "Björn Lindqvist" <bjourne@gmail.com> wrote:
> > The emacs-26.1-x86_64.zip file is really big. It contains a lot of
> > files which I wonder why they are necessary. Some examples
> >
> >     python2.7.exe
> >     gdbus.exe
> >     libgdk_pixbuf-2.0-0.dll
> >     include/jasper/
> >     include/GL
> >     include/gnutls AND include/openssl
> >     lib/systemd
> >     sqlite3200.dll
> >     lib/pkgconfig
> >     lib/cmake
> >     share/bash-completion
> >     share/vala
> >     ...
>
> Some people want the binary zip to include all the optional features
> that Emacs on Windows can support.

Fair enough. But what optional features are missing from the
-no-deps.zip file? GnuTLS support in eww? Maybe the name of the
emacs-26.1-x86_64.zip file could be changed to indicate that it is an
"all inclusive" package? Most users are probably fine with downloading
the smaller emacs-26.1-x86_64-no-deps.zip instead.

> > The emacs-26.1-x86_64-no-deps.zip installation is smaller, but still
> > double the size of the corresponding 24.5 installation. This seem to
> > be because all binaries now include debugging symbols. Some examples
> > of the size increases:
> >
> >     addpm.exe 577 kB => 2 282 kB
> >     ctags.exe 956 kB => 3 245 kB
> >     emacs.exe 8 989 kB => 121 740 kB
> >     emacs-24.5.exe 8 989 kB => 121 740 kB (emacs-26.1.exe)
>
> Stripping emacs.exe produces a 29MB file for Emacs 26.2.

But why is it four times bigger than in 24.5? Has so much new C code
been added to Emacs? Gcc's machine code generation certainly hasn't
gotten worse so that can't be it.

> We don't provide any shell scripts or batch files because the build on
> Posix systems doesn't.  Once again, it's hard to blame volunteers for
> using the build products as is, without adding any more work.

Sorry, I didn't mean to blame anyone! If the omission is lack of
volunteer time then I can look at the build and see if I can figure
it out. As I wrote in the other thread, I have some experience porting
Linux applications to Windows.

Perhaps I shouldn't have used "bloat" in the email subject line.

> > According to the thread on help-gnu-emacs Emacs binaries used to be
> > stripped of debugging symbols, but aren't anymore and that is what is
> > causing the size increase. I wonder if we can return that? Most
> > software ported to Windows, such as MinGW, strips debugging symbols so
> > it is customary. For most users they are useless because they don't
> > run emacs.exe in gdb.exe.
>
> It is a great help to have debug info when problems are reported that
> cause crashes and cannot be easily reproduced.

I don't think I've ever had Emacs on Windows crash on me. But if it did,
how would I get hold of the stack trace? Executables on Windows are
mostly run by clicking on their icons and that hides the standard input
and output.

> But if Phillip can afford prodicing separate debug info file for
> Emacs, we could have the cake and eat it, too.

Do you mean afford as in time or as in the Windows build is run on a
rented server?

> > On my machine it increases Emacs
> > start time by a second (although I don't know if that is caused by the
> > debugging symbols or if it is something else).
>
> It cannot be due to debug info, because that is not read when the
> program loads.

You are right. I stripped the emacs.exe and the startup slowdown
remains. But it could still be related to the debugging symbols if emacs
is compiled with -Og which is customary for debug builds. Because if you
compile it with more optimizations (-O2 or -O3), the debugging
symbols becomes less useful as stack frames disappear and
-fomit-frame-pointer makes it harder for gdb to inspect the stack.

> > It is also
> > aesthetically displeasing -- hackers like minimalism and hate bloat.
>
> FWIW, I think you the first one to complain about this.

Well, yes, but how many Windows users complained about the lack of
debugging symbols in Emacs 24? I must admit I have a hard time
formulating why I think avoiding bloat is important, it just seem
self-evident. For example, Visual Studio 2017 is over 2GB and that
irritates me because why does it need so much space for just an IDE and
a C/C++-compiler?!

Emacs is also supposed to be usable on old operating systems and old
hardware so saving disk space should be important imho.

> > And while on the subject of Windows packaging. How come there is no
> > MSI installer for Emacs? It shouldn't be to hard to put one together
> > and it would make Emacs a little easier to install for newbies.
>
> What tools to use to produce the binary distribution is entirely up to
> the person who does that.  And of course MSI is not free software.

The WiX Toolset which one can use to build MSIs is free software. I
don't know about the MSI format itself, but it would surprise me if
there are any legal hurdles since MSIs are supposed to be distributed by
ISVs.


--
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Björn Lindqvist <bjourne@gmail.com> writes:

> The emacs-26.1-x86_64.zip file is really big. It contains a lot of
> files which I wonder why they are necessary. Some examples
>
>     python2.7.exe
>     gdbus.exe
>     libgdk_pixbuf-2.0-0.dll
>     include/jasper/
>     include/GL
>     include/gnutls AND include/openssl
>     lib/systemd
>     sqlite3200.dll
>     lib/pkgconfig
>     lib/cmake
>     share/bash-completion
>     share/vala



Yes. I have produced these via a script which is simplistic. You can
find it at admin/nt/dist-build/build-dep-zips.py. It is basically the
transitive dependencies of all Emacs dependencies. It's undoubtly to
wide (including python for example as you point out).

This does give Emacs a fairly full msys install. It would be possible to
shrink it do the direct dependencies (i.e. the lib files), but decided
completeness made more sense.


>     ...
>
> The emacs-26.1-x86_64-no-deps.zip installation is smaller, but still
> double the size of the corresponding 24.5 installation. This seem to
> be because all binaries now include debugging symbols. Some examples
> of the size increases:
>
>     addpm.exe 577 kB => 2 282 kB
>     ctags.exe 956 kB => 3 245 kB
>     emacs.exe 8 989 kB => 121 740 kB
>     emacs-24.5.exe 8 989 kB => 121 740 kB (emacs-26.1.exe)
>
> The emacs.exe growth is especially annoying because the file is
> copied. This seems like poor packaging to me. Why not have an
> emacs.bat file calling emacs-26.1.exe and immediately save 121M?

Adding emacs.bat would require seperate logic for windows.


>
> According to the thread on help-gnu-emacs Emacs binaries used to be
> stripped of debugging symbols, but aren't anymore and that is what is
> causing the size increase. I wonder if we can return that? Most
> software ported to Windows, such as MinGW, strips debugging symbols so
> it is customary. For most users they are useless because they don't
> run emacs.exe in gdb.exe.

It is simply for me to remove the -g option from the windows build. This
would reduce the overall size of the build, to the values given here:

http://lists.gnu.org/archive/html/help-gnu-emacs/2019-04/msg00115.html

I am happy to do this (for Emacs-27 -- I do not want to change during
major release). But I would like feedback from people who either use or
handle bug reports for Emacs on Windows to let me know whether this
would break things.


> For people with limited disk space, metered internet or old hardware
> the new, bigger Emacs is a nuisance. On my machine it increases Emacs
> start time by a second (although I don't know if that is caused by the
> debugging symbols or if it is something else). It is also
> aesthetically displeasing -- hackers like minimalism and hate bloat.
>
> And while on the subject of Windows packaging. How come there is no
> MSI installer for Emacs? It shouldn't be to hard to put one together
> and it would make Emacs a little easier to install for newbies.

I haven't written one. There is an .exe installer for Emacs-27. I
haven't worked out how to do an MSI yet; if there is a good free package
for doing so, that can be used straight-forwardly within the current
build, I would be happy to add it.

Incidentally, the installer version with all deps is smaller than the
current version with deps, because the compression is better, which
addresses some of your "bandwidth" concerns.

Phil

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> Date: Wed, 17 Apr 2019 17:49:06 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Date: Wed, 17 Apr 2019 08:39:52 -0400
> > 
> > FWIW, I think saving a 100MB is worth the extra work.
> 
> We cannot make that decision, only Phillip can.

And btw, invoking through a batch file is in no way 100% transparent.
For example, I think users who start Emacs from MSYS Bash will have to
go through cmd.exe, because Bash cannot interpret batch files
directly.  So we will have to provide both a batch file and a Unix
shell script, at least.

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> And while on the subject of Windows packaging. How come there is no
>>> MSI installer for Emacs? It shouldn't be to hard to put one together
>>> and it would make Emacs a little easier to install for newbies.
>> What tools to use to produce the binary distribution is entirely up to the
>> person who does that.  And of course MSI is not free software.
>
> BTW, I seem to remember someone working on an "installer" for Emacs
> on Windows.  Does anyone know what came out of that?

IIRC it tanked when (L)GLP-mandated source file distribution came to the
scenary.

Creating an installer is quite easy and others did that on the past.
Providing the sources of the tools and libraries that Emacs depends on
makes the work orders of magnitude more difficult.

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> It is simply for me to remove the -g option from the windows build. This
> would reduce the overall size of the build, to the values given here:
>
> http://lists.gnu.org/archive/html/help-gnu-emacs/2019-04/msg00115.html
>
> I am happy to do this (for Emacs-27 -- I do not want to change during
> major release). But I would like feedback from people who either use or
> handle bug reports for Emacs on Windows to let me know whether this
> would break things.

There is a simple approach to this: `make install' as usual, package as
usual (without dependencies) but name that package
emacs-debug-blah-blah. Now run `strip` on Emacs binary directory and
proceed to do the normal packaging.

If the user wants to debug Emacs at the C level, he grabs
emacs-debug-blah-blah and unzips it on top of his current install.

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Björn Lindqvist <bjourne@gmail.com>
> Date: Wed, 17 Apr 2019 17:07:23 +0200
> Cc: emacs-devel@gnu.org
> 
> > Some people want the binary zip to include all the optional features
> > that Emacs on Windows can support.
> 
> Fair enough. But what optional features are missing from the
> -no-deps.zip file?

All of them.  Image support beyond XPM, SSL/TLS connections, HTML and
XML parsing, built-in decompression, built-in JSON support.

> Maybe the name of the
> emacs-26.1-x86_64.zip file could be changed to indicate that it is an
> "all inclusive" package? Most users are probably fine with downloading
> the smaller emacs-26.1-x86_64-no-deps.zip instead.

AFAIR, we've changed the names in the opposite direction because most
people wanted the full version.

> > Stripping emacs.exe produces a 29MB file for Emacs 26.2.
> 
> But why is it four times bigger than in 24.5?

Because the disk image includes a large array which is used only in
its small part.  We need this to be able to strip emacs.exe, something
that in previous versions required building a special binary.  The
actual footprint in memory is much smaller, between 12MB and 18MB
depending on the configuration.

You really shouldn't worry that much about disk space, the memory
footprint of a running process is a much more relevant measure of
bloat.

> Has so much new C code been added to Emacs?

See above: most of it is data, not code.  And it isn't loaded into the
running process.

> > It is a great help to have debug info when problems are reported that
> > cause crashes and cannot be easily reproduced.
> 
> I don't think I've ever had Emacs on Windows crash on me. But if it did,
> how would I get hold of the stack trace? Executables on Windows are
> mostly run by clicking on their icons and that hides the standard input
> and output.

Emacs writes the stack trace to a file, and the user manual explains
how to convert that into human-readable addresses.

> > But if Phillip can afford prodicing separate debug info file for
> > Emacs, we could have the cake and eat it, too.
> 
> Do you mean afford as in time or as in the Windows build is run on a
> rented server?

I have no idea what would that mean for Phillip, that's something he
will have to answer.  If needed, I can help by showing the commands
required to move the debug info into a separate file.

> You are right. I stripped the emacs.exe and the startup slowdown
> remains. But it could still be related to the debugging symbols if emacs
> is compiled with -Og which is customary for debug builds.

AFAIK, it was compiled with "-O2 -g3", not -Og.

> Because if you
> compile it with more optimizations (-O2 or -O3), the debugging
> symbols becomes less useful as stack frames disappear and
> -fomit-frame-pointer makes it harder for gdb to inspect the stack.

Having symbols even in an optimized build is better than not having
them.  And I don't think Phillip uses -fomit-frame-pointer, because
it's a net loss.

> Well, yes, but how many Windows users complained about the lack of
> debugging symbols in Emacs 24?

Without symbols some problems that are reported cannot be investigated
and fixed.

> I must admit I have a hard time formulating why I think avoiding
> bloat is important, it just seem self-evident.

Not disk space.  Bloat of memory footprint, yes.

> For example, Visual Studio 2017 is over 2GB and that irritates me
> because why does it need so much space for just an IDE and a
> C/C++-compiler?!

Well, shall we postpone the argument until Emacs gets anywhere near
2GB?

> Emacs is also supposed to be usable on old operating systems and old
> hardware

Which old systems are those?

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 17 Apr 2019 18:25:50 +0200
> 
> > It is simply for me to remove the -g option from the windows build. This
> > would reduce the overall size of the build, to the values given here:
> >
> > http://lists.gnu.org/archive/html/help-gnu-emacs/2019-04/msg00115.html
> >
> > I am happy to do this (for Emacs-27 -- I do not want to change during
> > major release). But I would like feedback from people who either use or
> > handle bug reports for Emacs on Windows to let me know whether this
> > would break things.
> 
> There is a simple approach to this: `make install' as usual, package as
> usual (without dependencies) but name that package
> emacs-debug-blah-blah. Now run `strip` on Emacs binary directory and
> proceed to do the normal packaging.

There's a "make install-strip" target for that.

Removing -g is not TRT, btw, as it will still leave some minimal
symbols.

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Some people want the binary zip to include all the optional features that
>> Emacs on Windows can support.  This zip includes all of the dependencies for
>> those optional features.  Arguably, some of the auxiliary files, like header
>> files and import libraries, could be omitted, but determining which ones are
>> required is a very non-trivial and time-consuming job, so I can understand
>> why Phillip, who volunteered to produce the binary zips, didn't do that.
>> This "one cannot fit all" problem is why we also have the bare-minimum zip
>> with only the dependencies that are absolutely required.
>
> Indeed.  But I wonder about licensing issues of some of those binaries:
> for those packages which are (L)GPL'd, do we distribute the
> corresponding source somewhere, like the license requires us to do?

https://ftp.gnu.org/gnu/emacs/windows/emacs-26/emacs-26-deps-mingw-w64-src.zip

Packaged at the same time as the binaries, which is about the best I can
do to keep the source the same as that which was used to produce the
source.


>
>>>     addpm.exe 577 kB => 2 282 kB
>>>     ctags.exe 956 kB => 3 245 kB
>>>     emacs.exe 8 989 kB => 121 740 kB
>>>     emacs-24.5.exe 8 989 kB => 121 740 kB (emacs-26.1.exe)
>> Stripping emacs.exe produces a 29MB file for Emacs 26.2.
>
> I wonder what caused the 9MB => 29MB jump between 24.5 and 26?

Probably the -static flag that I added. Otherwise, the 64bit file
required libpthread and (at the time) I had no ability to distribute
this because I didn't have a way of generating the source.


>> We don't provide any shell scripts or batch files because the build on Posix
>> systems doesn't.  Once again, it's hard to blame volunteers for using the
>> build products as is, without adding any more work.
>
> FWIW, I think saving a 100MB is worth the extra work.

It isn't extra work. I just change the compile options on my build
scripts. I genuinely have no idea why they are there in the first place;
perhaps I put them in because someone told me to, perhaps not.


> (but, indeed, using hard-links is the better option when available:
> I often just `rm src/emacs-[0-9]*` to get rid of old Emacs releases it'd
> be much trickier to do if src/emacs were a script that runs
> src/emacs-27.0.50.5).

It's also possible to turn on NTFS file compression which will remove
both this and lots of other duplications in the Emacs install. AFAIK,
this is a user option, however, although I might be able to do it with
the installer version for Emacs-27.

Phil

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> And while on the subject of Windows packaging. How come there is no
>>> MSI installer for Emacs? It shouldn't be to hard to put one together
>>> and it would make Emacs a little easier to install for newbies.
>> What tools to use to produce the binary distribution is entirely up to the
>> person who does that.  And of course MSI is not free software.
>
> BTW, I seem to remember someone working on an "installer" for Emacs
> on Windows.  Does anyone know what came out of that?

https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/emacs-27.0.50-snapshot-2019-01-17-x86_64-installer.exe

Will become standard for Emacs-27. I missed the Emacs-26 fork.

Phil

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Björn Lindqvist <bjourne@gmail.com> writes:

>> Some people want the binary zip to include all the optional features
>> that Emacs on Windows can support.
>
> Fair enough. But what optional features are missing from the
> -no-deps.zip file? GnuTLS support in eww?

Image support is the one most people would hit I think. I have no
evidence for this statement of course.

> Maybe the name of the emacs-26.1-x86_64.zip file could be changed to
> indicate that it is an "all inclusive" package?

No, I do not want to do this. I changed this from emacs-25 -> emacs-26
because I think most people would want it all to just work following the
clicking the obvious download link.

> Most users are probably fine with downloading the smaller
> emacs-26.1-x86_64-no-deps.zip instead.

I might be able to get this data from the ftp logs. Not sure how
reliable that is, however.

> I don't think I've ever had Emacs on Windows crash on me. But if it did,
> how would I get hold of the stack trace? Executables on Windows are
> mostly run by clicking on their icons and that hides the standard input
> and output.
>
>> But if Phillip can afford prodicing separate debug info file for
>> Emacs, we could have the cake and eat it, too.
>
> Do you mean afford as in time or as in the Windows build is run on a
> rented server?

My time. I am entirely happy to either not add the debug info, or to put
it in a different file. I know how to do the former not the latter.


>> > It is also
>> > aesthetically displeasing -- hackers like minimalism and hate bloat.
>>
>> FWIW, I think you the first one to complain about this.
>
> Well, yes, but how many Windows users complained about the lack of
> debugging symbols in Emacs 24?

My guess is none. It's why I am glad (and suggested) raising the issue
on Emacs-devel.

>> > And while on the subject of Windows packaging. How come there is no
>> > MSI installer for Emacs? It shouldn't be to hard to put one together
>> > and it would make Emacs a little easier to install for newbies.
>>
>> What tools to use to produce the binary distribution is entirely up to
>> the person who does that.  And of course MSI is not free software.
>
> The WiX Toolset which one can use to build MSIs is free software. I
> don't know about the MSI format itself, but it would surprise me if
> there are any legal hurdles since MSIs are supposed to be distributed by
> ISVs.

If you are willing to help develop a MSI for Emacs-27 then that would be
a good thing and, so long as building it can be relatively automated on
top of the current process, then I will be happy to release that for
Emacs-27 onward. This and code signing are the two things that I have
not got working for the Windows build, but would like to.

Phil

Re: Bloat in the Emacs Windows package

[Stefan Monnier]

>> Indeed.  But I wonder about licensing issues of some of those binaries:
>> for those packages which are (L)GPL'd, do we distribute the
>> corresponding source somewhere, like the license requires us to do?
>
> https://ftp.gnu.org/gnu/emacs/windows/emacs-26/emacs-26-deps-mingw-w64-src.zip
>
> Packaged at the same time as the binaries, which is about the best I can
> do to keep the source the same as that which was used to produce the
> source.

Perfect.

>>> Stripping emacs.exe produces a 29MB file for Emacs 26.2.
>> I wonder what caused the 9MB => 29MB jump between 24.5 and 26?
> Probably the -static flag that I added.  Otherwise, the 64bit file
> required libpthread and (at the time) I had no ability to distribute
> this because I didn't have a way of generating the source.

Makes sense.  Thank you.


        Stefan

Re: Bloat in the Emacs Windows package

[Stefan Monnier]

>>> Some people want the binary zip to include all the optional features
>>> that Emacs on Windows can support.
>> Fair enough. But what optional features are missing from the
>> -no-deps.zip file? GnuTLS support in eww?
> Image support is the one most people would hit I think. I have no
> evidence for this statement of course.

I guess the libraries required for the default build (i.e. the ones for
which you need to explicitly say `--without-foo` to build Emacs without
support for them) would be the main ones missing:
- image libraries
- gnutls: very important nowadays to support accessing ELPA repositories
  via HTTPS, sending Email via authenticated SMTP, reading email via
  IMAP, ...
- xml2, for HTML rendering when reading mail and in eww, and I think
  it's also used by various other packages.


-- Stefan

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Wed, 17 Apr 2019 18:39:49 +0100
> 
> > Fair enough. But what optional features are missing from the
> > -no-deps.zip file? GnuTLS support in eww?
> 
> Image support is the one most people would hit I think. I have no
> evidence for this statement of course.

It isn't just images.  TLS support is important for package.el,
something that is very popular these days.  People who want to use LSP
and related packages will want JSON support.

> I am entirely happy to either not add the debug info, or to put
> it in a different file. I know how to do the former not the latter.

I can tell how to do the latter, if you want.  It boils down to
running 2 objcopy commands.

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Óscar Fuentes <ofv@wanadoo.es> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> It is simply for me to remove the -g option from the windows build. This
>> would reduce the overall size of the build, to the values given here:
>>
>> http://lists.gnu.org/archive/html/help-gnu-emacs/2019-04/msg00115.html
>>
>> I am happy to do this (for Emacs-27 -- I do not want to change during
>> major release). But I would like feedback from people who either use or
>> handle bug reports for Emacs on Windows to let me know whether this
>> would break things.
>
> There is a simple approach to this: `make install' as usual, package as
> usual (without dependencies) but name that package
> emacs-debug-blah-blah. Now run `strip` on Emacs binary directory and
> proceed to do the normal packaging.
>
> If the user wants to debug Emacs at the C level, he grabs
> emacs-debug-blah-blah and unzips it on top of his current install.

Alas, that would produce a combinatorial explosion -- for Emacs-27 there
are already 6 binary downloads. Adding a debug version of these would
make 12.

Phil

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Wed, 17 Apr 2019 18:25:50 +0200
>> 
>> > It is simply for me to remove the -g option from the windows build. This
>> > would reduce the overall size of the build, to the values given here:
>> >
>> > http://lists.gnu.org/archive/html/help-gnu-emacs/2019-04/msg00115.html
>> >
>> > I am happy to do this (for Emacs-27 -- I do not want to change during
>> > major release). But I would like feedback from people who either use or
>> > handle bug reports for Emacs on Windows to let me know whether this
>> > would break things.
>> 
>> There is a simple approach to this: `make install' as usual, package as
>> usual (without dependencies) but name that package
>> emacs-debug-blah-blah. Now run `strip` on Emacs binary directory and
>> proceed to do the normal packaging.
>
> There's a "make install-strip" target for that.
>
> Removing -g is not TRT, btw, as it will still leave some minimal
> symbols.


Currently, I build with

"-O2 -static -g3"

I could change this to

-O2 -static -g0

When I did this, a binary that didn't really change size after running
strip (I don't have the exact figures for this, but could test). It
could also change this to

-O3 -static -g0

which would make things slightly faster, but I guess a large executable
(again I haven't tested). Or, I could "make install" to "make
install-strip".

None of these things would cost me anything other than changing one
file. I am happy what ever. Just tell me which one works best.

Phil

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
>> Date: Wed, 17 Apr 2019 18:39:49 +0100
>> 
>> > Fair enough. But what optional features are missing from the
>> > -no-deps.zip file? GnuTLS support in eww?
>> 
>> Image support is the one most people would hit I think. I have no
>> evidence for this statement of course.
>
> It isn't just images.  TLS support is important for package.el,
> something that is very popular these days.  People who want to use LSP
> and related packages will want JSON support.
>
>> I am entirely happy to either not add the debug info, or to put
>> it in a different file. I know how to do the former not the latter.
>
> I can tell how to do the latter, if you want.  It boils down to
> running 2 objcopy commands.

So many options!

Again, happy to do whatever can be fitted into my shell script that I
use to build the zip files.

Phil

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

Phillip Lord <phillip.lord@russet.org.uk> writes:

>> There is a simple approach to this: `make install' as usual, package as
>> usual (without dependencies) but name that package
>> emacs-debug-blah-blah. Now run `strip` on Emacs binary directory and
>> proceed to do the normal packaging.
>>
>> If the user wants to debug Emacs at the C level, he grabs
>> emacs-debug-blah-blah and unzips it on top of his current install.
>
> Alas, that would produce a combinatorial explosion -- for Emacs-27 there
> are already 6 binary downloads. Adding a debug version of these would
> make 12.

Not really. One debug package containing the binaries with debug info is
enough. The idea is to be able to unzip the debug package over an
existing install and replace the non-debug binaries with the debug
binaries.

For bonus points, build those debug binaries with optimizations turned
off :-)

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: bjourne@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 18 Apr 2019 17:05:14 +0100
> 
> > I can tell how to do the latter, if you want.  It boils down to
> > running 2 objcopy commands.
> 
> So many options!
> 
> Again, happy to do whatever can be fitted into my shell script that I
> use to build the zip files.

Not sure if that's a yes, but anyway...

Given an unstripped emacs.exe, you produce the stripped emacs.exe and
the separate debug info file like this:

  objcopy --only-keep-debug emacs.exe emacs.debug
  strip -g emacs.exe
  objcopy --add-gnu-debuglink=emacs.debug emacs.exe

When unpacked, the emacs.debug file should install into the bin/.debug
directory of the Emacs installation.

HTH

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: bjourne@gmail.com,  emacs-devel@gnu.org
>> Date: Thu, 18 Apr 2019 17:05:14 +0100
>> 
>> > I can tell how to do the latter, if you want.  It boils down to
>> > running 2 objcopy commands.
>> 
>> So many options!
>> 
>> Again, happy to do whatever can be fitted into my shell script that I
>> use to build the zip files.
>
> Not sure if that's a yes, but anyway...
>
> Given an unstripped emacs.exe, you produce the stripped emacs.exe and
> the separate debug info file like this:
>
>   objcopy --only-keep-debug emacs.exe emacs.debug
>   strip -g emacs.exe
>   objcopy --add-gnu-debuglink=emacs.debug emacs.exe
>
> When unpacked, the emacs.debug file should install into the bin/.debug
> directory of the Emacs installation.

Would this impact on emacs-26.2.exe also or would I need to things like:

objcopy --only-keep-debug emacs.exe emacs.debug
strip -g emacs.exe
strip -g emacs-26.2.exe
objcopy --add-gnu-debuglink=emacs.debug emacs.exe
objcopy --add-gnu-debuglink=emacs.debug emacs-26.2.exe




To summarise my feelings about the thread so far:

 - I think most normal users don't need debug symbols, so I would be
   minded to remove them (or not put them). I don't know why I have
   added "-g3" to the default options. People who know what to do with
   debug symbols are likely to be able to build Emacs for
   themselves.

 - We currently install emacs.exe and emacs-26.2.exe. I think we should
   continue doing this because it is how we do it on other
   platforms. The disk space requirement is small (and will be smaller
   if  we remove debug symbols). It probably adds little to the
   download bandwith (because of zip) and will add nothing with the .exe
   installer for Emacs-27. And NTFS compression fixes the problem (and
   more) for those who really care about space (and they are probably
   using it already).

I am happy to be corrected here if I am missing some unintended
consequence, or if you disagree with my justifications.


Unanswered questions for me:

 - If we remove debug symbols, why not do -O3 which may produce some
   performance benefit?
 - Is it better to do "-g0" which I believe stops adding debug symbols,
   or switch "make install" for "make install-strip"?


Doing any of these things (adding -O3, -g0 or using make install-strip)
are trivial to implement and cost me nothing in ongoing maintenance.

Phil


 

Re: Bloat in the Emacs Windows package

[Björn Lindqvist]

Den ons 17 apr. 2019 kl 18:18 skrev Eli Zaretskii <eliz@gnu.org>:
> > Date: Wed, 17 Apr 2019 17:49:06 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: emacs-devel@gnu.org
> >
> > > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > > Date: Wed, 17 Apr 2019 08:39:52 -0400
> > >
> > > FWIW, I think saving a 100MB is worth the extra work.
> >
> > We cannot make that decision, only Phillip can.
>
> And btw, invoking through a batch file is in no way 100% transparent.
> For example, I think users who start Emacs from MSYS Bash will have to
> go through cmd.exe, because Bash cannot interpret batch files
> directly.  So we will have to provide both a batch file and a Unix
> shell script, at least.

That's true so a batch file is no good. But isn't the runemacs.exe
file already handling the indirection? If that file is compiled with
CHOOSE_NEWEST_EXE defined, then it will automagically choose the most
recent emacs*.exe if multiple version are installed in the same
directory.

Personally, I never run the emacs.exe nor emacs-xy.z.exe directly
because it opens up a useless cmd.exe window. Instead I use
runemacs.exe from the Start menu and emacsclientw.exe from the
Explorer shell.


--
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> To summarise my feelings about the thread so far:
>
>  - I think most normal users don't need debug symbols,

I agree.

>    so I would be
>    minded to remove them (or not put them). I don't know why I have
>    added "-g3" to the default options. People who know what to do with
>    debug symbols are likely to be able to build Emacs for
>    themselves.

Not really. Building Emacs is much harder than installing gdb, execute
emacs under gdb and obtain an stack trace.

Debug symbols are useful, and that is the reason why GNU/Linux
distributions offer them as a separate package.

>  - We currently install emacs.exe and emacs-26.2.exe. I think we should
>    continue doing this because it is how we do it on other
>    platforms.

As explained on a previous post, IMO the reasoning behind that practice
makes little sense for Windows.

>    The disk space requirement is small (and will be smaller if we
>    remove debug symbols). It probably adds little to the download
>    bandwith (because of zip) and will add nothing with the .exe
>    installer for Emacs-27. And NTFS compression fixes the problem (and
>    more) for those who really care about space (and they are probably
>    using it already).

Creating symlinks on Windows (pre-10, IIRC) require administrator
privileges. Ditto for enabling NTFS compression.

About what is small and what isn't: that's a personal opinion. I'm mildly
annoyed about the fact that yesterday's MB are today's GB, without a
proportional increase in functionality. Also, I recently started using
an Android tablet with 32 GB storage and I really appreciate how the
Termux guys manage to produce packages that are significantly smaller
than those on desktop GNU/Linux. There are low-end computers in stores
with 32 GB SSDs. And people use old hardware too.

> I am happy to be corrected here if I am missing some unintended
> consequence, or if you disagree with my justifications.
>
>
> Unanswered questions for me:
>
>  - If we remove debug symbols, why not do -O3 which may produce some
>    performance benefit?

-O3 does not necessarily mean better performance. My bet is that the
difference will be insignificant for Emacs. I've seen plenty of cases
where -O3 was measurably worse than -O2.

Re: Bloat in the Emacs Windows package

[Björn Lindqvist]

Den ons 17 apr. 2019 kl 18:32 skrev Eli Zaretskii <eliz@gnu.org>:
> > > Some people want the binary zip to include all the optional features
> > > that Emacs on Windows can support.
> >
> > Fair enough. But what optional features are missing from the
> > -no-deps.zip file?
>
> All of them.  Image support beyond XPM, SSL/TLS connections, HTML and
> XML parsing, built-in decompression, built-in JSON support.

Are all those dependencies necessary? Even on gnu/linux distros (like
Arch Linux, Ubuntu, ...) when you install Emacs it doesn't require all
that. And on that OS other packages likely to be installed already
depends on some dependencies (like libgdk-pixbuf2.0) so the
"dependency burden" is shared among lots of software.

> > > Stripping emacs.exe produces a 29MB file for Emacs 26.2.
> >
> > But why is it four times bigger than in 24.5?
>
> Because the disk image includes a large array which is used only in
> its small part.

Apparently it's the same on gnu/linux.

> > Because if you compile it with more optimizations (-O2 or -O3),
> > the debugging symbols becomes less useful as stack frames
> > disappear and -fomit-frame-pointer makes it harder for gdb to
> > inspect the stack.
>
> Having symbols even in an optimized build is better than not having
> them.

But my point is that you cannot optimize the build completely while
still having usable debugging symbols. If you use -O3 then gcc's
inlining of stack frames and other transformations make the debugging
symbol data close to useless. Even with -O2 you get lots of
"<optimized out>" parameters and some missing stack frames. I.e the
worst of both worlds; neither full optimizations nor full debugging
symbols.

I dunno if it would make a difference for Emacs, but other projects
I've benchmarked ran noticeably faster when compiled with -O3 over
-O2. Especially CPU bound tasks like elisp compilation would perhaps
run faster.

> And I don't think Phillip uses -fomit-frame-pointer, because it's a
> net loss.

Why? In my limited experience it enables gcc to produce significantly
better code on the register starved x86 32 bit architecture.

> > I must admit I have a hard time formulating why I think avoiding
> > bloat is important, it just seem self-evident.
>
> Not disk space.  Bloat of memory footprint, yes.

I disagree and think that both disk space and memory footprint is
important to keep small. Most gnu/linux distributors think the same
because they strip the emacs executables even though it makes it
harder to debug crashes.

> > Emacs is also supposed to be usable on old operating systems and old
> > hardware
>
> Which old systems are those?

According to the GNU Emacs FAQ for MS Windows they are Windows 98 and
Windows NT 4.0 through to Windows 8.1. It doesn't make sense claiming
Windows 98 support without not also ensuring that the software fits on
a hard disk a Windows 98 user would use. Hence the claim, to me,
implies that there is an upper bound on how large the Emacs
installation can become.


--
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Björn Lindqvist]

Den ons 17 apr. 2019 kl 19:27 skrev Phillip Lord <phillip.lord@russet.org.uk>:
> It's also possible to turn on NTFS file compression which will remove
> both this and lots of other duplications in the Emacs install. AFAIK,
> this is a user option, however, although I might be able to do it with
> the installer version for Emacs-27.

True, but remember that not all Windows users have administrator
access on their machines. For example, a loooong way back in
university we students had to create a Java project using NetBeans (or
was it Eclipse? IntelliJ? I don't remember) on Windows. I didn't like
NetBeans so I instead installed Emacs (21 I think) to my H: drive
which was 500M of personal space allocated to each student.

A few years later while on vacation in southern Europe I had to write a
(paper) letter to some government agency. I didn't have a laptop with
me, so I instead went to an internet cafe, downloaded Emacs into a
temporary directory, wrote the letter in Emacs and printed it.

Now in each of these cases I didn't have to use Emacs. I could have
used NetBeans as the instructor wanted and I could have written my
letter in write.exe. But it felt nice that Emacs had my back and that
I could work the way I wanted. If Emacs would have been to slow to
download, or taken up to much disk space, I wouldn't have been able
to.

Since then, both bandwidth and disk space has gotten cheaper. But I
bet there are still lots of situations where Emacs' smallness is
important. Especially in poorer parts of the world.


--
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: bjourne@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 18 Apr 2019 22:19:26 +0100
> 
> >   objcopy --only-keep-debug emacs.exe emacs.debug
> >   strip -g emacs.exe
> >   objcopy --add-gnu-debuglink=emacs.debug emacs.exe
> >
> > When unpacked, the emacs.debug file should install into the bin/.debug
> > directory of the Emacs installation.
> 
> Would this impact on emacs-26.2.exe also or would I need to things like:
> 
> objcopy --only-keep-debug emacs.exe emacs.debug
> strip -g emacs.exe
> strip -g emacs-26.2.exe
> objcopy --add-gnu-debuglink=emacs.debug emacs.exe
> objcopy --add-gnu-debuglink=emacs.debug emacs-26.2.exe

There is no magic: if emacs.exe and emacs-26.2.exe are two separate
files (not hard links to the same file data), then yes, you will need
the additional commands above.  Sorry for not saying that explicitly.

> To summarise my feelings about the thread so far:
> 
>  - I think most normal users don't need debug symbols, so I would be
>    minded to remove them (or not put them). I don't know why I have
>    added "-g3" to the default options. People who know what to do with
>    debug symbols are likely to be able to build Emacs for
>    themselves.

Are we only talking about official releases, or do you include
development snapshots in the above reasoning?  Snapshots should
include the symbols IMO, to facilitate more efficient debugging of
problems reported for them.  The assumption is that people who install
snapshots are more cognizant about debugging and are less "normal"
than those who download official releases, in the sense that they
prefer efficient bug reports to disk space savings.

> Unanswered questions for me:
> 
>  - If we remove debug symbols, why not do -O3 which may produce some
>    performance benefit?

IME, -O3 is useless for Emacs, probably because the expensive inner
loops are very large.  It does produce a larger executable, which is
another downside.  It is also less well tested, since thedefault build
uses -O2.

That said, my metrics were done years ago, and it's possible that
nowadays -O3 performs better.  If we want to explore this, someone
should try building Emacs with -O3 and producing some measurements for
GCC versions 7 and up to 9.  And I don't think this is Windows
specific: if -O3 produces worthy benefits, we might consider using it
in production builds on all platforms.

>  - Is it better to do "-g0" which I believe stops adding debug symbols,
>    or switch "make install" for "make install-strip"?

I'd say the latter, because 'install-strip' is the official GNU way of
getting a stripped binary.

Thanks.

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Björn Lindqvist <bjourne@gmail.com>
> Date: Fri, 19 Apr 2019 00:02:55 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> > And btw, invoking through a batch file is in no way 100% transparent.
> > For example, I think users who start Emacs from MSYS Bash will have to
> > go through cmd.exe, because Bash cannot interpret batch files
> > directly.  So we will have to provide both a batch file and a Unix
> > shell script, at least.
> 
> That's true so a batch file is no good. But isn't the runemacs.exe
> file already handling the indirection? If that file is compiled with
> CHOOSE_NEWEST_EXE defined, then it will automagically choose the most
> recent emacs*.exe if multiple version are installed in the same
> directory.

First, runemacs.exe is by default _not_ compiled with
CHOOSE_NEWEST_EXE defined.  I don't think we should turn that feature
on by default, because the result might be unpredictable: no one can
guarantee that the newest emacs*.exe is what the user wants.  E.g.,
what if some older version is rebuilt and re-installed, or if the user
downloads and installs an executable from another source whose
timestamp is older?  This kind of stuff is not for the faint at heart,
IMO, and that's why it's turned off.

> Personally, I never run the emacs.exe nor emacs-xy.z.exe directly
> because it opens up a useless cmd.exe window.

Not if you invoke it from a cmd.exe window.  And that is an important,
albeit relatively rarely used feature, because it allows several
important use cases, such as:

  . using Emacs in -batch mode
  . starting Emacs from emacsclient
  . starting "emacs -Q" to test if some problem happens then
  . if Emacs crashes, the backtrace is printed on the console as well,
    which is more visible than when you invoke via runemacs

> Instead I use runemacs.exe from the Start menu and emacsclientw.exe
> from the Explorer shell.

There are use cases beyond those covered by what you describe.  E.g.,
emacsclientw is inappropriate when Emacs is to be called from a
program that must wait for Emacs to finish editing, like when it is
used as EDITOR from the various VCS programs.

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Björn Lindqvist <bjourne@gmail.com>
> Date: Fri, 19 Apr 2019 01:44:09 +0200
> Cc: emacs-devel@gnu.org
> 
> > > Fair enough. But what optional features are missing from the
> > > -no-deps.zip file?
> >
> > All of them.  Image support beyond XPM, SSL/TLS connections, HTML and
> > XML parsing, built-in decompression, built-in JSON support.
> 
> Are all those dependencies necessary?

The answer depends on what you intend to do with Emacs.  No one can
know that in advance.  Emacs on Windows is built in a way that not
having some dependency available at run time is not fatal, it just
means that the respective features will be unavailable, and will
trigger error messages when invoked.  So you can selectively delete
the features you don't need, or start with none and selectively
install those you do need.  However, many people don't want to go
through this non-trivial procedure of finding the optional packages
which are compatible with the Emacs binary you have (there's the
32-bit vs 64-bit issue, and then there are different versions of each
optional package), and there was a popular demand for a single large
installation with all the dependencies guaranteed to work.  So that's
what Phillip does.

IOW, the current arrangement more or less reflects what people asked
us.  Your needs seem to be special, but then why not install the
MSYS/MinGW development environment and build your own Emacs as you see
fit, installing the optional support packages that you think you will
need?  It isn't hard, and the detailed instructions are in
nt/INSTALL.W64 (assuming you want to build with MinGW64).

> Even on gnu/linux distros (like Arch Linux, Ubuntu, ...) when you
> install Emacs it doesn't require all that. And on that OS other
> packages likely to be installed already depends on some dependencies
> (like libgdk-pixbuf2.0) so the "dependency burden" is shared among
> lots of software.

GNU/Linux distros have a tool to manage installed packages.  We don't.
GNU/Linux distros are separate operations dedicated to distributing
packages in coherent ways, something we aren't and can never be.  If
you want to install Emacs that way, I suggest to explore the MSYS2
project, perhaps it has Emacs packaged more to your liking.  This here
is the upstream Emacs project, and providing a binary package for
Windows is already much more than we do for other platforms.  We
generally produce only the sources.

> > > > Stripping emacs.exe produces a 29MB file for Emacs 26.2.
> > >
> > > But why is it four times bigger than in 24.5?
> >
> > Because the disk image includes a large array which is used only in
> > its small part.
> 
> Apparently it's the same on gnu/linux.

No, it isn't.  The unexec machinery is platform specific.  Emacs 27
moves to portable dumping, which will allow us to remove that array.
That makes an unstripped emacs.exe about 85MB on my system, down to
just 8.5MB when stripped.  The price is an additional 9.5MB dump file
that needs to be around when Emacs starts, for the grand total of
18MB.

> > Having symbols even in an optimized build is better than not having
> > them.
> 
> But my point is that you cannot optimize the build completely while
> still having usable debugging symbols. If you use -O3 then gcc's
> inlining of stack frames and other transformations make the debugging
> symbol data close to useless. Even with -O2 you get lots of
> "<optimized out>" parameters and some missing stack frames. I.e the
> worst of both worlds; neither full optimizations nor full debugging
> symbols.

That is true, but recent versions of GCC and GDB are much better in
these circumstances, and still manage to produce useful information in
optimized builds.  It is admittedly less informative than in
unoptimized builds, but it is _much_ better than no information at
all, because at least most of the functions in a backtrace are shown
with their names, not as "??".

> I dunno if it would make a difference for Emacs, but other projects
> I've benchmarked ran noticeably faster when compiled with -O3 over
> -O2. Especially CPU bound tasks like elisp compilation would perhaps
> run faster.

Feel free to benchmark Emacs and present the results.  My (years-old)
benchmarks told a different story, but maybe things have changed with
the newer GCC versions.  Most GNU project stil ship Makefile's that
use -O2 in production builds, so Emacs is not alone in this,
definitely not on Windows.

> > And I don't think Phillip uses -fomit-frame-pointer, because it's a
> > net loss.
> 
> Why?

Because the backtraces are much less informative, and because modern
64-bit processors have enough registers even without having one more.

> > Which old systems are those?
> 
> According to the GNU Emacs FAQ for MS Windows they are Windows 98 and
> Windows NT 4.0 through to Windows 8.1. It doesn't make sense claiming
> Windows 98 support without not also ensuring that the software fits on
> a hard disk a Windows 98 user would use.

We have the no-deps package for people whose disk space is at premium.
And anyway, are you saying that a 100MB file doesn't fit on a typical
disk of a Windows 9X system?  There's a limit to exaggeration when you
want to make a point; if you exceed that limit, your arguments run the
risk that people will not regard them as serious.

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Björn Lindqvist <bjourne@gmail.com>
> Date: Fri, 19 Apr 2019 02:02:31 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> Since then, both bandwidth and disk space has gotten cheaper. But I
> bet there are still lots of situations where Emacs' smallness is
> important. Especially in poorer parts of the world.

Of course.  But those situations need special-purpose packages, and
each one of those will need a volunteer dedicated to produce and
support it.  The popular demand clearly shifted to having the
comprehensive packages with everything included.

Feel free to volunteer to produce a bare-minimum Emacs package for
such situation.

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Since then, both bandwidth and disk space has gotten cheaper. But I
>> bet there are still lots of situations where Emacs' smallness is
>> important. Especially in poorer parts of the world.
>
> Of course.  But those situations need special-purpose packages, and
> each one of those will need a volunteer dedicated to produce and
> support it.  The popular demand clearly shifted to having the
> comprehensive packages with everything included.
>
> Feel free to volunteer to produce a bare-minimum Emacs package for
> such situation.

We have the no-deps package, but with dozens of MB of debug info and a
duplicated large binary. I doubt this benefits 0.1% of the Emacs user
base. And that benefit is just a minor convenience for them.

OTOH, the bigger the file to download, the less people will get it.
People that will use something else. For a political (or social, if you
prefer) project as GNU/Emacs, this is not acceptable when you can
significantly reduce the download with a few trivial changes (1).

I live on Western Europe and know plenty of people who access the
Internet through metered and/or slow connections.

And then we have the issue with general bloat in IT and the associated
waste. I feel morally obligued to not contribute to that waste.


1. Using a better compression method, such as 7z, probably will reduce
the download significantly.

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Are all those dependencies necessary?
>
> The answer depends on what you intend to do with Emacs.

Phillip goes to great lengths to be helpful, but IMO he is distributing
too much stuff. There are plenty of dlls on the Emacs /bin directory
with potential to break things. It would break my setup for sure,
because it contains the tcl/tk dlls that will interfere with what I have
installed.

>> According to the GNU Emacs FAQ for MS Windows they are Windows 98 and
>> Windows NT 4.0 through to Windows 8.1. It doesn't make sense claiming
>> Windows 98 support without not also ensuring that the software fits on
>> a hard disk a Windows 98 user would use.
>
> We have the no-deps package for people whose disk space is at premium.
> And anyway, are you saying that a 100MB file doesn't fit on a typical
> disk of a Windows 9X system?  There's a limit to exaggeration when you
> want to make a point; if you exceed that limit, your arguments run the
> risk that people will not regard them as serious.

106 MB is the zip file. 366 MB unzipped. If the Windows 98 machine runs
on a surviving machine of the same era, that's a big chunk of a typical
hard drive.

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Fri, 19 Apr 2019 16:17:55 +0200
> 
> 106 MB is the zip file. 366 MB unzipped. If the Windows 98 machine runs
> on a surviving machine of the same era, that's a big chunk of a typical
> hard drive.

But not big enough to exhaust the drive, I'm sure.

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> Would this impact on emacs-26.2.exe also or would I need to things like:
>> 
>> objcopy --only-keep-debug emacs.exe emacs.debug
>> strip -g emacs.exe
>> strip -g emacs-26.2.exe
>> objcopy --add-gnu-debuglink=emacs.debug emacs.exe
>> objcopy --add-gnu-debuglink=emacs.debug emacs-26.2.exe
>
> There is no magic: if emacs.exe and emacs-26.2.exe are two separate
> files (not hard links to the same file data), then yes, you will need
> the additional commands above.  Sorry for not saying that explicitly.

From what you are saying, emacs.exe and emacs-26.2.exe *will be* hard
links before they are zipped, but not after.


>> To summarise my feelings about the thread so far:
>> 
>>  - I think most normal users don't need debug symbols, so I would be
>>    minded to remove them (or not put them). I don't know why I have
>>    added "-g3" to the default options. People who know what to do with
>>    debug symbols are likely to be able to build Emacs for
>>    themselves.
>
> Are we only talking about official releases, or do you include
> development snapshots in the above reasoning?  Snapshots should
> include the symbols IMO, to facilitate more efficient debugging of
> problems reported for them.

I am currently not treating snapshots and release builds differently at
all. When I took over the windows builds, there were no snapshots to my
knowledge. It's possible that is why I put debug symbols in and meant to
take them out but didn't.

> The assumption is that people who install snapshots are more cognizant
> about debugging and are less "normal" than those who download official
> releases, in the sense that they prefer efficient bug reports to disk
> space savings.

Indeed. Perhaps, here-in, lies the solution.

I currently have snapshots for master only. I could produce these for
both master and release branch and build these with debug info; while
tagged releases would get them without.

The risk here is that snapshots might get behind release builds, since
I'd consider the release to be more important than the snapshot when a
new tag is made.


>> Unanswered questions for me:
>> 
>>  - If we remove debug symbols, why not do -O3 which may produce some
>>    performance benefit?
>
> IME, -O3 is useless for Emacs, probably because the expensive inner
> loops are very large.  It does produce a larger executable, which is
> another downside.  It is also less well tested, since thedefault build
> uses -O2.
>
> That said, my metrics were done years ago, and it's possible that
> nowadays -O3 performs better.  If we want to explore this, someone
> should try building Emacs with -O3 and producing some measurements for
> GCC versions 7 and up to 9.  And I don't think this is Windows
> specific: if -O3 produces worthy benefits, we might consider using it
> in production builds on all platforms.


At the moment, I specify -O2 specifically in my build. I guess this is
because if I specify any CFLAGS I need to give them all. Still, if this
is not windows specific I will wait till Emacs over all changes.


>>  - Is it better to do "-g0" which I believe stops adding debug symbols,
>>    or switch "make install" for "make install-strip"?
>
> I'd say the latter, because 'install-strip' is the official GNU way of
> getting a stripped binary.

This I will do.

Phil

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> Are all those dependencies necessary?
>>
>> The answer depends on what you intend to do with Emacs.
>
> Phillip goes to great lengths to be helpful, but IMO he is distributing
> too much stuff. There are plenty of dlls on the Emacs /bin directory
> with potential to break things. It would break my setup for sure,
> because it contains the tcl/tk dlls that will interfere with what I have
> installed.

I think that there are two plausible options here. The first is the one
that Chris from Emacs-W64 took. He put the DLLs for the direct
dependencies in (libjpg, libgif and so on) and some others. That's nice,
but it doesn't fulfil all parts of Emacs functionality -- for example,
M-x find-dired doesn't work, nor does flyspell because they lack find
and aspell.

The other option is to do what I have done; I have taken Emacs direct
dependencies and pulled in all the transistive dependencies according to
MSYS2, right or wrong. So M-x find-dired does work (as do images and as
does package.el). In the process you also get tcl/tk and, indeed,
python. Some things still do not work (so auctex can't compile because
of no latex).

I choose the later because I didn't want to hand-maintain a list of
dependencies for myself. This would be a significant task and one that
it would be hard to perform well, especially in the absence of a test
set for all of Emacs' functionality.

I am happy to consider a more minimal distribution of Emacs but I need
to know a way of producing this minimal dependency list. The code for
this is in admin/nt/dist-build. I welcome any amendments to this, but it
has to be automatic. In the mean time, you have the choice of a
dependency free version for when faced with issues like tcl/tk.

Phil

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Of course.  But those situations need special-purpose packages, and
>> each one of those will need a volunteer dedicated to produce and
>> support it.  The popular demand clearly shifted to having the
>> comprehensive packages with everything included.
>>
>> Feel free to volunteer to produce a bare-minimum Emacs package for
>> such situation.
>
> We have the no-deps package, but with dozens of MB of debug info and a
> duplicated large binary. I doubt this benefits 0.1% of the Emacs user
> base. And that benefit is just a minor convenience for them.

I made the no-deps package to address your use case -- when you have the
dependencies already. Size was not my primary concern.


> OTOH, the bigger the file to download, the less people will get it.
> People that will use something else. For a political (or social, if you
> prefer) project as GNU/Emacs, this is not acceptable when you can
> significantly reduce the download with a few trivial changes (1).
>
> I live on Western Europe and know plenty of people who access the
> Internet through metered and/or slow connections.
>
> And then we have the issue with general bloat in IT and the associated
> waste. I feel morally obligued to not contribute to that waste.

I agree that this is morally correct; but then so is producing free
software which is competitive with non-free software. I do not believe
that an Emacs incapable of loading jpgs or using its own package manager
securely fulfils that objective.

> 1. Using a better compression method, such as 7z, probably will reduce
> the download significantly.

Yes, and this is why the Emacs-27.exe installer uses lzma. It is,
indeed, smaller with deps than the zip version without. But, I missed
the emacs-26 branch.

Phil

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

phillip.lord@russet.org.uk (Phillip Lord) writes:

>> We have the no-deps package, but with dozens of MB of debug info and a
>> duplicated large binary. I doubt this benefits 0.1% of the Emacs user
>> base. And that benefit is just a minor convenience for them.
>
> I made the no-deps package to address your use case -- when you have the
> dependencies already. Size was not my primary concern.

In my view the no-deps is geared towards those who just want to edit
text. It is great to have image support, but I'm sure many Emacs users
don't really care. A different thing is the lack of grep/find, though.

>> And then we have the issue with general bloat in IT and the associated
>> waste. I feel morally obligued to not contribute to that waste.
>
> I agree that this is morally correct; but then so is producing free
> software which is competitive with non-free software. I do not believe
> that an Emacs incapable of loading jpgs or using its own package manager
> securely fulfils that objective.

See above. However, I was referring to the debug info and emacs.exe
duplication, not to the accompanying software that goes with the
no-no-deps.

Re: Bloat in the Emacs Windows package

[Stefan Monnier]

> I am currently not treating snapshots and release builds differently at
> all.  When I took over the windows builds, there were no snapshots to my
> knowledge.  It's possible that is why I put debug symbols in and meant to
> take them out but didn't.

Indeed, the snapshots should be distributed with full debug info
(even if some users don't appreciate it).


        Stefan

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: bjourne@gmail.com,  emacs-devel@gnu.org
> Date: Mon, 22 Apr 2019 21:40:39 +0100
> 
> >> objcopy --only-keep-debug emacs.exe emacs.debug
> >> strip -g emacs.exe
> >> strip -g emacs-26.2.exe
> >> objcopy --add-gnu-debuglink=emacs.debug emacs.exe
> >> objcopy --add-gnu-debuglink=emacs.debug emacs-26.2.exe
> >
> > There is no magic: if emacs.exe and emacs-26.2.exe are two separate
> > files (not hard links to the same file data), then yes, you will need
> > the additional commands above.  Sorry for not saying that explicitly.
> 
> >From what you are saying, emacs.exe and emacs-26.2.exe *will be* hard
> links before they are zipped, but not after.

If they are hard links before the above commands, you don't need
separate commands for each executable.  When unzipped, they will both
have the same debuglink section.

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I am currently not treating snapshots and release builds differently at
>> all.  When I took over the windows builds, there were no snapshots to my
>> knowledge.  It's possible that is why I put debug symbols in and meant to
>> take them out but didn't.
>
> Indeed, the snapshots should be distributed with full debug info
> (even if some users don't appreciate it).


So, currently we have:

CFLAGS="-O2 -static -g3"

In future, we should have for snapshots

CFLAGS="-O2 -static -g3"

and build with "make install", while for tagged releases (including
release candidates) I will use:

CFLAGS="-O2 -static"

and build with "make install-strip" (I can't see the point in doing -g3
and make install-strip).

I'll also build some snapshots for release branches (as well as master)
so that there will be a build with debug info to be distributed on
alpha.gnu.org. This will be mostly be at a slightly different commit
from the release; so, sometimes no use for debugging a new bug
introduced into a recent release, although, I can always build a new
snapshot.

This will happen for Emacs-27, as I would rather not alter the packaging
during a major release cycle. Independently of this, I'll see if I can
work up an MSI installer.

Phil

Re: Bloat in the Emacs Windows package

[Robert Pluim]

>>>>> On Tue, 23 Apr 2019 11:01:39 +0100, phillip.lord@russet.org.uk (Phillip Lord) said:
    Phillip> This will happen for Emacs-27, as I would rather not
    Phillip> alter the packaging during a major release
    Phillip> cycle. Independently of this, I'll see if I can work up
    Phillip> an MSI installer.

Thereʼs this one:

<https://chaoticlab.io/posts/emacs-26-2-msi>

Robert

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Tue, 23 Apr 2019 11:01:39 +0100, phillip.lord@russet.org.uk
> (Phillip Lord) said:
>     Phillip> This will happen for Emacs-27, as I would rather not
>     Phillip> alter the packaging during a major release
>     Phillip> cycle. Independently of this, I'll see if I can work up
>     Phillip> an MSI installer.
>
> Thereʼs this one:
>
> <https://chaoticlab.io/posts/emacs-26-2-msi>


Thanks! I'd found out about this in January but, to my shame, forgotten
about it.

I am in communication with Artem now, hopefully I can get something
working on the main line from this.

Phil

Re: Bloat in the Emacs Windows package

[Phillip Lord]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> I am currently not treating snapshots and release builds differently at
>>> all.  When I took over the windows builds, there were no snapshots to my
>>> knowledge.  It's possible that is why I put debug symbols in and meant to
>>> take them out but didn't.
>>
>> Indeed, the snapshots should be distributed with full debug info
>> (even if some users don't appreciate it).
>
>
> So, currently we have:
>
> CFLAGS="-O2 -static -g3"
>
> In future, we should have for snapshots
>
> CFLAGS="-O2 -static -g3"
>
> and build with "make install", while for tagged releases (including
> release candidates) I will use:
>
> CFLAGS="-O2 -static"
>
> and build with "make install-strip" (I can't see the point in doing -g3
> and make install-strip).
>
> I'll also build some snapshots for release branches (as well as master)
> so that there will be a build with debug info to be distributed on
> alpha.gnu.org. This will be mostly be at a slightly different commit
> from the release; so, sometimes no use for debugging a new bug
> introduced into a recent release, although, I can always build a new
> snapshot.


I have now trialled there and there are versions on alpha

https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/

The "emacs-27.0.50" files are built like a release build, while the
"-snapshot" ones are built like a snapshot build.

In terms of the download size, -no-deps drops from 107M to 46M which is
the most extreme drop. The installer version drops by only 10M because
it's lzma compressed. Uncompressed sizes will be proportionately
larger.

Happy if people want to give these a trial to see if there is any
functional or otherwise noticable difference between the two. I update
the build scripts on master in a short while if there are no complaints.

Phil

Re: Bloat in the Emacs Windows package

[Óscar Fuentes]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> I have now trialled there and there are versions on alpha
>
> https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/
>
> The "emacs-27.0.50" files are built like a release build, while the
> "-snapshot" ones are built like a snapshot build.
>
> In terms of the download size, -no-deps drops from 107M to 46M which is
> the most extreme drop.

Quite a difference. I'm sure there are plenty of users all around the
world that will appreciate this change. Thank you.

BTW, seeing emacs.exe with less than 5MB gives a fuzzy feeling. Even
emacs.exe + emacs.pdmp is less than 12 MB.

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Óscar Fuentes <ofv@wanadoo.es> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> I have now trialled there and there are versions on alpha
>>
>> https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/
>>
>> The "emacs-27.0.50" files are built like a release build, while the
>> "-snapshot" ones are built like a snapshot build.
>>
>> In terms of the download size, -no-deps drops from 107M to 46M which is
>> the most extreme drop.
>
> Quite a difference. I'm sure there are plenty of users all around the
> world that will appreciate this change. Thank you.
>
> BTW, seeing emacs.exe with less than 5MB gives a fuzzy feeling. Even
> emacs.exe + emacs.pdmp is less than 12 MB.


Happy to do it. It doesn't look like it was a deliberate decision
anyway.

Phil

Re: Bloat in the Emacs Windows package

[Björn Lindqvist]

Den fre 26 apr. 2019 kl 18:41 skrev Phillip Lord <phillip.lord@russet.org.uk>:
> I have now trialled there and there are versions on alpha
>
> https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/
>
> The "emacs-27.0.50" files are built like a release build, while the
> "-snapshot" ones are built like a snapshot build.
>
> In terms of the download size, -no-deps drops from 107M to 46M which is
> the most extreme drop. The installer version drops by only 10M because
> it's lzma compressed. Uncompressed sizes will be proportionately
> larger.

The emacs.exe file has shrunk from 121 mb to 6.2 mb. That is pretty, pretty
good. Thanks a lot! I've only lightly tested it so far, but it seems
that nothing
isbroken. eww doesn't work due to missing libxml2, but it didn't work in
emacs 24 either. There is an 10mb emacs.pdmp file in
libexec\emacs\27.0.50\x86_64-w64-mingw32 which must be some leftover from the
old build process. Even with that file, the install is roughly as
small as it was in
version 24.5.


-- 
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Phillip Lord]

Björn Lindqvist <bjourne@gmail.com> writes:

> Den fre 26 apr. 2019 kl 18:41 skrev Phillip Lord <phillip.lord@russet.org.uk>:
>> I have now trialled there and there are versions on alpha
>>
>> https://alpha.gnu.org/gnu/emacs/pretest/windows/emacs-27/
>>
>> The "emacs-27.0.50" files are built like a release build, while the
>> "-snapshot" ones are built like a snapshot build.
>>
>> In terms of the download size, -no-deps drops from 107M to 46M which is
>> the most extreme drop. The installer version drops by only 10M because
>> it's lzma compressed. Uncompressed sizes will be proportionately
>> larger.
>
> The emacs.exe file has shrunk from 121 mb to 6.2 mb. That is pretty, pretty
> good.

Most of that is because the lisp data is dumped independently now. I
think about 50Mb is because of the lack of debug symbols. The duplicated
executables are no longer an issue, though, since the data is only
stored once.


> Thanks a lot! I've only lightly tested it so far, but it seems that
> nothing isbroken. eww doesn't work due to missing libxml2, but it
> didn't work in emacs 24 either.

libxml2 is a dependency, so it should be there in the versions with
dependencies. If it isn't working, then it's a bug.


> There is an 10mb emacs.pdmp file in
> libexec\emacs\27.0.50\x86_64-w64-mingw32 which must be some leftover
> from the old build process.

No, the pdmp file is where all the dumped lisp data went. My release
builds (but not the snapshots) are all clean, so leftovers shouldn't be
possible.

Phil

Re: Bloat in the Emacs Windows package

[Björn Lindqvist]

Den lör 27 apr. 2019 kl 20:17 skrev Phillip Lord <phillip.lord@russet.org.uk>:
> > There is an 10mb emacs.pdmp file in
> > libexec\emacs\27.0.50\x86_64-w64-mingw32 which must be some leftover
> > from the old build process.
>
> No, the pdmp file is where all the dumped lisp data went. My release
> builds (but not the snapshots) are all clean, so leftovers shouldn't be
> possible.

Cool. I didn't know that. Although I tried to remove it and emacs
still works fine.
But it feels like it starts a little slower. Perhaps emacs could
regenerate that file
if it detects it is missing.


-- 
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Björn Lindqvist <bjourne@gmail.com>
> Date: Fri, 3 May 2019 03:06:33 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> I tried to remove it and emacs still works fine. But it feels like
> it starts a little slower.

Of course.  That's the whole reason for having that file.

> Perhaps emacs could regenerate that file if it detects it is
> missing.

I don't think Emacs should write any files without user's say-so.
There is a function you can invoke to produce a dump file, so an
unusual situation such as this one does have a solution.

Re: Bloat in the Emacs Windows package

[Björn Lindqvist]

> > I tried to remove it and emacs still works fine. But it feels like
> > it starts a little slower.
>
> Of course.  That's the whole reason for having that file.
>
> > Perhaps emacs could regenerate that file if it detects it is
> > missing.
>
> I don't think Emacs should write any files without user's say-so.
> There is a function you can invoke to produce a dump file, so an
> unusual situation such as this one does have a solution.

Wouldn't it be more efficient if each user had their own .pdmp file?
For example,  I never use gnus but I use markdown-mode and erc
all the time so, for me, it would be better if the latter two packages were
part of the dump image but not the former.


-- 
mvh/best regards Björn Lindqvist

Re: Bloat in the Emacs Windows package

[Eli Zaretskii]

> From: Björn Lindqvist <bjourne@gmail.com>
> Date: Fri, 3 May 2019 20:38:09 +0200
> Cc: Phillip Lord <phillip.lord@russet.org.uk>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	emacs-devel@gnu.org
> 
> Wouldn't it be more efficient if each user had their own .pdmp file?
> For example,  I never use gnus but I use markdown-mode and erc
> all the time so, for me, it would be better if the latter two packages were
> part of the dump image but not the former.

The plan is to allow that, yes.  We are not there yet: currently,
re-dumping causes some incorrect behavior.

But regardless of that, the standard pdump file, the one which is
produced when Emacs is built from sources, should always be there, so
not having it is unusual.

Re: Bloat in the Emacs Windows package

[Stefan Monnier]

>> Wouldn't it be more efficient if each user had their own .pdmp file?
>> For example,  I never use gnus but I use markdown-mode and erc
>> all the time so, for me, it would be better if the latter two packages were
>> part of the dump image but not the former.

Gnus is not part of the pdmp file: the normal pdmp file only includes
those packages which we think will be used by a majority of sessions.


        Stefan