Avoid duplicate emacs.exe / emacs-$version.exe

Avoid duplicate emacs.exe / emacs-$version.exe

[Juan José García-Ripoll]

[-- Attachment #1: Type: text/plain, Size: 1923 bytes --]

Right now in my hard disk I have two copies of statically linked Emacs.
~/emacs-27/bin/emacs.exe
~/emacs-27/bin/emacs-27.0.90.exe (let me call it emacs-$version.exe)

Each of them is 128Mb large and waste space. I understand the latter is
only meant to be used when the version is required. runemacs.exe and
tools actually seem to rely on emacs.exe, because if I delete
emacs-$version.exe it still works fine.  Same goes for earlier releases.

I would suggest one of these options

a) Remove emacs-$version.exe during installation.

b) Same as a), but install a script emacs-$version.cmd containing just two
lines
    @echo off
    %~dp0\emacs.exe %*

c) Instead, remove emacs.exe and leave only emacs-$version.exe Make emacs.cmd a
script
    @echo off
    %~dp0\emacs-$version.exe %*
where $version is replaced with the current version. We also have to change
runemacs.exe to look for emacs-$version.exe

Option a) only invokes a change in Makefile.in. Option b) includes that change,
plus the creation of the script. This is in a patch attached to this
message. Option c) would be the most intrusive. I personally do not like it
because it binds a general utility, runemacs.exe, to a particular version.

After the patch:
juanj@DESKTOP-3A8AAJ0 MINGW64 ~/emacs-build/build/juanjo
$ du -smc ~/emacs-27/*
135     /c/Users/juanj/emacs-27/bin
1       /c/Users/juanj/emacs-27/include
1       /c/Users/juanj/emacs-27/lib
17      /c/Users/juanj/emacs-27/libexec
98      /c/Users/juanj/emacs-27/share
249     total

Before the patch:
juanj@DESKTOP-3A8AAJ0 MINGW64 ~/emacs-build/build/juanjo
$ du -smc ~/emacs-27/*
252     /c/Users/juanj/emacs-27/bin
1       /c/Users/juanj/emacs-27/include
1       /c/Users/juanj/emacs-27/lib
17      /c/Users/juanj/emacs-27/libexec
98      /c/Users/juanj/emacs-27/share
366     total

A 32% reduction in size.

--
Juan José García Ripoll
http://juanjose.garciaripoll.com
http://quinfog.hbar.es

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: emacs-cmd.diff --]
[-- Type: text/x-patch, Size: 1644 bytes --]

diff --git a/Makefile.in b/Makefile.in
index 67e15cf..67c14f2 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -499,10 +499,13 @@ install-arch-dep:
 endif
 	-chmod 755 "$(DESTDIR)${bindir}/$(EMACSFULL)"
 ifndef NO_BIN_LINK
+ifeq ($(NTDIR),nt)
+	cd "$(DESTDIR)${bindir}" && mv "$(EMACSFULL)" "$(EMACS)"
+else
 	rm -f "$(DESTDIR)${bindir}/$(EMACS)"
 	cd "$(DESTDIR)${bindir}" && $(LN_S_FILEONLY) "$(EMACSFULL)" "$(EMACS)"
 endif
-else
+endif
 	subdir=${ns_appresdir}/site-lisp && ${write_subdir}
 	rm -rf ${ns_appresdir}/share
 endif
diff --git a/nt/Makefile.in b/nt/Makefile.in
index 6bdf824..7af6e78 100644
--- a/nt/Makefile.in
+++ b/nt/Makefile.in
@@ -167,7 +167,7 @@ LINK_CFLAGS =
 ## Unused.
 CPP_CFLAGS = ${BASE_CFLAGS} ${PROFILING_CFLAGS} ${CPPFLAGS} ${CFLAGS}
 
-all: ${EXE_FILES}
+all: ${EXE_FILES} emacs-$(version).cmd
 
 .PHONY: all
 
@@ -194,6 +194,7 @@ install:
 	for file in ${INSTALLABLES} ; do \
 	  $(INSTALL_PROGRAM) $(INSTALL_STRIP) $${file} "$(DESTDIR)${bindir}"/`echo $${file} | sed -e 's/${EXEEXT}$$//' -e '$(TRANSFORM)'`${EXEEXT} ; \
 	done
+	$(INSTALL_PROGRAM) emacs-$(version).cmd "$(DESTDIR)${bindir}"
 	${MKDIR_P} "$(DESTDIR)${datadir}/emacs/$(version)"
 	$(INSTALL_DATA) ${srcdir}/README.W32 "$(DESTDIR)${datadir}/emacs/$(version)"
 
@@ -251,6 +252,10 @@ cmdproxy${EXEEXT}:
 runemacs${EXEEXT}: ${srcdir}/runemacs.c $(EMACSRES)
 	$(AM_V_CCLD)$(CC) ${ALL_CFLAGS} $^ -mwindows -o $@
 
+emacs-$(version).cmd:
+	echo '@echo off' > $@
+	echo '%~dp0\emacs.exe %*' >> $@
+
 ## Also used in ../src/Makefile.
 emacs.res ../src/emacs.res: emacs.rc ${srcdir}/icons/emacs.ico \
   ${srcdir}/icons/hand.cur ${srcdir}/$(EMACS_MANIFEST)

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Juan José García-Ripoll
>  <juanjose.garciaripoll@gmail.com>
> Date: Sat, 28 Mar 2020 17:53:00 +0100
> 
> Right now in my hard disk I have two copies of statically linked Emacs.

(This is a point, but Emacs is not linked statically, at least not by
default.  The large size is mostly due to debug info, and you can
strip it if you want, although I don't recommend doing so for a
pretest version, because you cannot produce meaningful backtraces from
a stripped binary.)

> ~/emacs-27/bin/emacs.exe
> ~/emacs-27/bin/emacs-27.0.90.exe (let me call it emacs-$version.exe)
> 
> Each of them is 128Mb large and waste space.

No, they don't waste space, at least not by default.  When you install
Emacs, the installation procedure produces a hard link with another
name to the same file data.  These two names are just 2 different
names that refer to the same disk space.

> I would suggest one of these options
> 
> a) Remove emacs-$version.exe during installation.
> 
> b) Same as a), but install a script emacs-$version.cmd containing just two
> lines
>     @echo off
>     %~dp0\emacs.exe %*
> 
> c) Instead, remove emacs.exe and leave only emacs-$version.exe Make emacs.cmd a
> script
>     @echo off
>     %~dp0\emacs-$version.exe %*
> where $version is replaced with the current version. We also have to change
> runemacs.exe to look for emacs-$version.exe

None of this is needed, since there's no waste of space, when our
installation procedure is used.

> After the patch:
> juanj@DESKTOP-3A8AAJ0 MINGW64 ~/emacs-build/build/juanjo
> $ du -smc ~/emacs-27/*
> 135     /c/Users/juanj/emacs-27/bin
> 1       /c/Users/juanj/emacs-27/include
> 1       /c/Users/juanj/emacs-27/lib
> 17      /c/Users/juanj/emacs-27/libexec
> 98      /c/Users/juanj/emacs-27/share
> 249     total
> 
> Before the patch:
> juanj@DESKTOP-3A8AAJ0 MINGW64 ~/emacs-build/build/juanjo
> $ du -smc ~/emacs-27/*
> 252     /c/Users/juanj/emacs-27/bin
> 1       /c/Users/juanj/emacs-27/include
> 1       /c/Users/juanj/emacs-27/lib
> 17      /c/Users/juanj/emacs-27/libexec
> 98      /c/Users/juanj/emacs-27/share
> 366     total

This means your installation procedure is modified, or maybe you
installed a binary someone else produced, in which case the archive
used to package the binaries didn't support hard links.  You can
restore the hard link by removing onhe of the copies and making a hard
link to the remaining copy under the other name.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Juan José García-Ripoll
>  <juanjose.garciaripoll@gmail.com>
> Date: Sat, 28 Mar 2020 17:53:00 +0100
> 
> Right now in my hard disk I have two copies of statically linked Emacs.

(This is a secondary point, but Emacs is not linked statically, at
least not by default.  The large size is mostly due to debug info, and
you can strip it if you want, although I don't recommend doing so for
a pretest version, because you cannot produce meaningful backtraces
from a stripped binary.)

> ~/emacs-27/bin/emacs.exe
> ~/emacs-27/bin/emacs-27.0.90.exe (let me call it emacs-$version.exe)
> 
> Each of them is 128Mb large and waste space.

No, they don't waste space, at least not by default.  When you install
Emacs, the installation procedure produces a hard link with another
name to the same file data.  These two names are just 2 different
names that refer to the same disk space.

> I would suggest one of these options
> 
> a) Remove emacs-$version.exe during installation.
> 
> b) Same as a), but install a script emacs-$version.cmd containing just two
> lines
>     @echo off
>     %~dp0\emacs.exe %*
> 
> c) Instead, remove emacs.exe and leave only emacs-$version.exe Make emacs.cmd a
> script
>     @echo off
>     %~dp0\emacs-$version.exe %*
> where $version is replaced with the current version. We also have to change
> runemacs.exe to look for emacs-$version.exe

None of this is needed, since there's no waste of space, when our
installation procedure is used.

> After the patch:
> juanj@DESKTOP-3A8AAJ0 MINGW64 ~/emacs-build/build/juanjo
> $ du -smc ~/emacs-27/*
> 135     /c/Users/juanj/emacs-27/bin
> 1       /c/Users/juanj/emacs-27/include
> 1       /c/Users/juanj/emacs-27/lib
> 17      /c/Users/juanj/emacs-27/libexec
> 98      /c/Users/juanj/emacs-27/share
> 249     total
> 
> Before the patch:
> juanj@DESKTOP-3A8AAJ0 MINGW64 ~/emacs-build/build/juanjo
> $ du -smc ~/emacs-27/*
> 252     /c/Users/juanj/emacs-27/bin
> 1       /c/Users/juanj/emacs-27/include
> 1       /c/Users/juanj/emacs-27/lib
> 17      /c/Users/juanj/emacs-27/libexec
> 98      /c/Users/juanj/emacs-27/share
> 366     total

This means your installation procedure is modified, or maybe you
installed a binary someone else produced, in which case the archive
used to package the binaries didn't support hard links.  You can
restore the hard link by removing onhe of the copies and making a hard
link to the remaining copy under the other name.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> Right now in my hard disk I have two copies of statically linked Emacs.
>
> (This is a secondary point, but Emacs is not linked statically, at
> least not by default.  The large size is mostly due to debug info, and
> you can strip it if you want, although I don't recommend doing so for
> a pretest version, because you cannot produce meaningful backtraces
> from a stripped binary.)

The pretests should be stripped actually. I use make install for
snapshot builds. The ones with an actual version number are make
install-strip'd.


> This means your installation procedure is modified, or maybe you
> installed a binary someone else produced, in which case the archive
> used to package the binaries didn't support hard links.  You can
> restore the hard link by removing onhe of the copies and making a hard
> link to the remaining copy under the other name.

Incidentally, I build, run and (sort of) test the Windows distribution
under NTFS compression which all works fine. If these circumstances, I
think, even the difference between two files or hard-linked is moot.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Juan José García-Ripoll]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Juan José García-Ripoll
>>  <juanjose.garciaripoll@gmail.com>
>> Date: Sat, 28 Mar 2020 17:53:00 +0100
>> 
>> Right now in my hard disk I have two copies of statically linked Emacs.
>
> (This is a point, but Emacs is not linked statically, at least not by
> default.  The large size is mostly due to debug info, and you can
> strip it if you want, although I don't recommend doing so for a
> pretest version, because you cannot produce meaningful backtraces from
> a stripped binary.)

I am talking about the official archives *.zip, not the *.exe install
file. I assumed "statically linked" because of the size and because of
the dependencies they have been built against -- e.g. libpng, libtiff,
etc.-- but from your message I deduce those libraries are loaded by
Emacs only when available.

> No, they don't waste space, at least not by default.  When you install
> Emacs, the installation procedure produces a hard link with another
> name to the same file data.  These two names are just 2 different
> names that refer to the same disk space. [...]
> This means your installation procedure is modified, or maybe you
> installed a binary someone else produced, in which case the archive
> used to package the binaries didn't support hard links.  You can
> restore the hard link by removing onhe of the copies and making a hard
> link to the remaining copy under the other name.

I am reporting figures that come from either (a) the official release
from ftp.gnu.org, (b) the prerelease *.zip files from alpha.gnu.org and
(c) the official build process as reported in emacs-27/nt.

- For emacs 26.3, using Windows' explorer to unzip the *-no-deps.zip
  file creates two identical files, emacs.exe and emacs-26.3.exe. MSYS's
  "du" reports a total of 277Mb usage. Windows directory properties
  reports the same figure for "size" and "size occupied in disk"
  (apologies, my copy of Windows is in Spanish and I do not know what
  text is used in other locales)

- Doing the same thing with a command line utility, unzip.exe, produces
  the same result.

- For emacs 27.0.90, using Windows' explorer or unzip.exe to inflate the
  *-no-deps.zip creates two identical files, but this time 6Mb in
  size. Once more, both are two separate files, though now total space is just
  13Mb (!)

- Building from emacs-27 branch (Savannah's git), following the
  instructions from nt/ (i.e. configure + make install), creates two
  identical files, emacs.exe and emacs-27.0.90.exe. Space usage is as in
  26.3, about 123Mb per executable.

So I am using stock files, never my own copies. I am using also standard
procedures. I do not understand why the executable sizes differ so much
between release, prerelease and built from sources. However, there are
no symbolic links happening at all. Indeed, MSYS's "ln" as used in the
autoconf build process does not seem to work: it just copies the file.

$ ln -sf foo faa
$ cat faa
Foo
$ echo Faa > foo
$ cat faa
Foo

My computer is an up-to-date Windows 10 (64-bit) installation using
NTFS, btw. My versions of MSYS and Mingw64 are also pretty recent.

So maybe you are discussing what happens in Linux or Mac systems?

Cheers,

-- 
Juan José García Ripoll
http://juanjose.garciaripoll.com
http://quinfog.hbar.es

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Juan José García-Ripoll]

Phillip Lord <phillip.lord@russet.org.uk> writes:
> Incidentally, I build, run and (sort of) test the Windows distribution
> under NTFS compression which all works fine. If these circumstances, I
> think, even the difference between two files or hard-linked is moot.

Not everyone has the luxury of commanding full configuration of their
computers. Some have to install software at the user level, without
administrator priviledges to do that configuration, or in a hardened
virtual machine or a computer that is not really up to the task of
handling everything with compression. So, while space nowadays seems
"free", if small changes can tidy up a piece of sofware, why not
implement them?

Best,

-- 
Juan José García Ripoll
http://juanjose.garciaripoll.com
http://quinfog.hbar.es

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Juan José García-Ripoll <juanjose.garciaripoll@gmail.com> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>> Incidentally, I build, run and (sort of) test the Windows distribution
>> under NTFS compression which all works fine. If these circumstances, I
>> think, even the difference between two files or hard-linked is moot.
>
> Not everyone has the luxury of commanding full configuration of their
> computers. Some have to install software at the user level, without
> administrator priviledges to do that configuration, or in a hardened
> virtual machine or a computer that is not really up to the task of
> handling everything with compression. So, while space nowadays seems
> "free", if small changes can tidy up a piece of sofware, why not
> implement them?


I think we all carry around the prejudices of our childhood. I taught
myself to program a while back. I often take a slow, cautious approach
with memory and space. My students tend not to at least not when talking
about less than 100Mb and increasing Gb. I used to shake my head in
despair, but then I realised that their code was simpler, cleaner and
faster than mine.

It's good that you are picking around the Windows build and I would like
to incorporate some of the changes; but not at the cost of (too much)
increased complexity in the source code. My build files have got too
complex already.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Juan José García-Ripoll <juanjose.garciaripoll@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>>> From: Juan José García-Ripoll
>>>  <juanjose.garciaripoll@gmail.com>


> So I am using stock files, never my own copies. I am using also standard
> procedures. I do not understand why the executable sizes differ so much
> between release, prerelease and built from sources.

It depends whether debugging is added or not. release and pre-release
are built with `make install-strip`, snapshots are built with `make
install`. Emacs-26 was built with `make install` regardless, and also
was a dumped binary so contained all the stuff now in the pdump file.

The result of all of this is that the executable installer for Emacs-27
is now less than half the size of "no-deps" download for Emacs-26. Of
course, this doesn't impact on space on disk. For release version of
Emacs-27, the difference between a hard linked or duplicated emacs.exe
is 6Mb.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Juan José García-Ripoll]

Eli Zaretskii <eliz@gnu.org> writes:
> (This is a secondary point, but Emacs is not linked statically, at
> least not by default.

Just a side note: the build-zips.sh script that makes the Windows
distribution files always uses the flag -static when building emacs. I
understand this is done so that Emacs does not require copies of the C,
mingw and other libraries.

So, for the record, perhaps a more accurate statement is that Emacs in
Windows is always statically built, except for some libraries, which are
loaded dynamically by the executable itself, not through the OS linking
process

decompress.c:  HMODULE library = w32_delayed_load (Qzlib);
gnutls.c:  if (!(library = w32_delayed_load (Qgnutls)))
image.c:  if (!(library = w32_delayed_load (Qxpm)))
image.c:  if (!(library = w32_delayed_load (Qpng)))
image.c:  if (!(library = w32_delayed_load (Qjpeg)))
image.c:  if (!(library = w32_delayed_load (Qtiff)))
image.c:  if (!(library = w32_delayed_load (Qgif)))
image.c:  if (!(glib = w32_delayed_load (Qglib))
image.c:      || !(gobject = w32_delayed_load (Qgobject))
image.c:      || !(giolib = w32_delayed_load (Qgio))
image.c:      || !(gdklib = w32_delayed_load (Qgdk_pixbuf))
image.c:      || !(library = w32_delayed_load (Qsvg)))
json.c:  HMODULE library = w32_delayed_load (Qjson);
lcms.c:  HMODULE library = w32_delayed_load (Qlcms2);
xml.c:      if (!(library = w32_delayed_load (Qlibxml2)))

-- 
Juan José García Ripoll
http://juanjose.garciaripoll.com
http://quinfog.hbar.es

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Juan José García-Ripoll]

Phillip Lord <phillip.lord@russet.org.uk> writes:
> It's good that you are picking around the Windows build and I would like
> to incorporate some of the changes; but not at the cost of (too much)
> increased complexity in the source code. My build files have got too
> complex already.

De-duplicating the emacs.exe file is between 3 and 6 lines of fix,
depending on whether we want to keep emacs-$version.exe around. It does
not touch your build files, just the standard install process. But I
appreciate that your builds with install-strip are way leaner now.

Best,

-- 
Juan José García Ripoll
http://juanjose.garciaripoll.com
http://quinfog.hbar.es

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: Juan José García-Ripoll
>  <juanjose.garciaripoll@gmail.com>,
>   emacs-devel@gnu.org
> Date: Sat, 28 Mar 2020 20:13:09 +0000
> 
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Right now in my hard disk I have two copies of statically linked Emacs.
> >
> > (This is a secondary point, but Emacs is not linked statically, at
> > least not by default.  The large size is mostly due to debug info, and
> > you can strip it if you want, although I don't recommend doing so for
> > a pretest version, because you cannot produce meaningful backtraces
> > from a stripped binary.)
> 
> The pretests should be stripped actually. I use make install for
> snapshot builds. The ones with an actual version number are make
> install-strip'd.

Given the size, I don't think they are stripped.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Juan José García-Ripoll
>  <juanjose.garciaripoll@gmail.com>
> Date: Sat, 28 Mar 2020 21:41:51 +0100
> 
> > No, they don't waste space, at least not by default.  When you install
> > Emacs, the installation procedure produces a hard link with another
> > name to the same file data.  These two names are just 2 different
> > names that refer to the same disk space. [...]
> > This means your installation procedure is modified, or maybe you
> > installed a binary someone else produced, in which case the archive
> > used to package the binaries didn't support hard links.  You can
> > restore the hard link by removing onhe of the copies and making a hard
> > link to the remaining copy under the other name.
> 
> I am reporting figures that come from either (a) the official release
> from ftp.gnu.org, (b) the prerelease *.zip files from alpha.gnu.org and
> (c) the official build process as reported in emacs-27/nt.

AFAIK, the ZIP archive indeed doesn't support hard links, at least not
on MS-Windows.  But you can recreate the hard link locally, if you
want, after unzipping the archive.

> - Building from emacs-27 branch (Savannah's git), following the
>   instructions from nt/ (i.e. configure + make install), creates two
>   identical files, emacs.exe and emacs-27.0.90.exe. Space usage is as in
>   26.3, about 123Mb per executable.

They aren't 2 files, they are 2 _names_.  Look at the link count of
each file, and you will see 2, not 1.

> So I am using stock files, never my own copies. I am using also standard
> procedures. I do not understand why the executable sizes differ so much
> between release, prerelease and built from sources. However, there are
> no symbolic links happening at all. Indeed, MSYS's "ln" as used in the
> autoconf build process does not seem to work: it just copies the file.

I didn't say symbolic links, I said hard links.

> $ ln -sf foo faa

Not "ln -s", just "ln".

> So maybe you are discussing what happens in Linux or Mac systems?

It happens on all supported systems.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Juan José García-Ripoll
>  <juanjose.garciaripoll@gmail.com>
> Date: Sun, 29 Mar 2020 00:36:12 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> > (This is a secondary point, but Emacs is not linked statically, at
> > least not by default.
> 
> Just a side note: the build-zips.sh script that makes the Windows
> distribution files always uses the flag -static when building emacs.

I don't think the build should use that.  It is certainly not the
default.

> I understand this is done so that Emacs does not require copies of
> the C, mingw and other libraries.

Some of that.  The C library cannot be linked in statically, because
Windows doesn't come with static versions of the MSVCRT.

> So, for the record, perhaps a more accurate statement is that Emacs in
> Windows is always statically built

Not "always", only if you use the scripts provided by Phillip.  The
default build procedure doesn't use -static.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Juan José García-Ripoll]

So let me pause here and retrace this discussion, because I feel like
the cat that is chasing the red pointer with you guys.

- My original question was: what is the official build process for
  making standalone Emasc packages, so as to reproduce the content of
  the official distros and understand what goes in. That is in a
  separate thread.

- The reply to that answer was to look into admin/nt/ where there are
  instructions and two scripts, one for pulling dependencies and another
  one for the build

- After reproducing those steps, I end up with zips and installers. The
  current official distribution of 26.3 has no installer. Only zips. The
  instruction is to unzip those packages.

- Unzipping either the official distro or the ZIPs that are built with
  the instructions I was provided creates two identical executables, not
  a hard link. This can be checked with Windows's fsutil (see below)

- Phillip's statement is that really this does not matter because the
  pretest builds show stripping the executables brings their size down
  to 6Mb. I verified this and I verified that this is true also for the
  26.3 build.

- Eli's claiming that i) those executables are dynamically linked and
  that ii) emacs.exe and emacs-27.0.90.exe (or emacs-26.3.exe in the
  previous release) are not two files, but two hard-linked copies.

Now, there are two different lines of thought that diverge: i) the issue
of dynamic linking, ii) the issue of whether the two executables are the
same.

Regarding i)
- I have verified that the distro build process reported in admin/nt
  starts with CFLAGS="-O2 -static" by default. Some options change it
  to CFLAGS="-O3 -static -g", but the -static flag is never dropped.

Regarding ii)
- I have verified that the files are not hard links. Actually, runing
  "configure" with the official sources produces this in config.status
            S["LN_S_FILEONLY"]="/bin/ln -s"
  so there is no way this is going to create hard links. This is also
  true in the case of the builds created with the procedure from
  emacs/admin/nt This makes sense for msys/mingw because in general
  there may be out there file systems which are still FAT-based and do
  not support hard links.

Now I get this reply, which I merge from a different line in the thread

Eli Zaretskii <eliz@gnu.org> writes:
>> Just a side note: the build-zips.sh script that makes the Windows
>> distribution files always uses the flag -static when building emacs.
> I don't think the build should use that.  It is certainly not the
> default.

So where is the default? Is the default build process not the one in
emacs/admin/nt or, in case one builds from sources without creating zip
files, the one in emacs/nt/? I am reporting data from running those two
procedures. If you are claiming I am not doing the standard thing, where
is the standard thing?

Apologies if I may sometimes sound rude, but I am committing my workflow
and part of my group's to Emacs for various reasons (org-mode, latex, ebib,
etc) and I would like to contribute make the upcoming releases work
nicely with our systems.

Best,

--- checking hard links ---

Official 26.3 distro:
$ fsutil hardlink list emacs.exe
\Users\juanj\emacs-26.3\bin\emacs.exe
$ fsutil hardlink list emacs-26.3.exe
\Users\juanj\emacs-26.3\bin\emacs-26.3.exe

Pretest 27.0.90 installer:
$ fsutil hardlink list emacs.exe
\Users\juanj\emacs-pretest\bin\emacs.exe
$ fsutil hardlink list emacs-27.0.90.exe
\Users\juanj\emacs-pretest\bin\emacs-27.0.90.exe

Emacs built from sources:
$ fsutil hardlink list emacs.exe
\Users\juanj\emacs-27\bin\emacs.exe
$ fsutil hardlink list emacs-27.0.90.exe
\Users\juanj\emacs-27\bin\emacs-27.0.90.exe

etc, etc.


-- 
Juan José García Ripoll
http://juanjose.garciaripoll.com
http://quinfog.hbar.es

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: Juan José García-Ripoll
>>  <juanjose.garciaripoll@gmail.com>,
>>   emacs-devel@gnu.org
>> Date: Sat, 28 Mar 2020 20:13:09 +0000
>> 
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Right now in my hard disk I have two copies of statically linked Emacs.
>> >
>> > (This is a secondary point, but Emacs is not linked statically, at
>> > least not by default.  The large size is mostly due to debug info, and
>> > you can strip it if you want, although I don't recommend doing so for
>> > a pretest version, because you cannot produce meaningful backtraces
>> > from a stripped binary.)
>> 
>> The pretests should be stripped actually. I use make install for
>> snapshot builds. The ones with an actual version number are make
>> install-strip'd.
>
> Given the size, I don't think they are stripped.


6.4Mb? And this is what the source says for the build. I know no other
way of checking.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Juan José García-Ripoll <juanjose.garciaripoll@gmail.com> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>> It's good that you are picking around the Windows build and I would like
>> to incorporate some of the changes; but not at the cost of (too much)
>> increased complexity in the source code. My build files have got too
>> complex already.
>
> De-duplicating the emacs.exe file is between 3 and 6 lines of fix,
> depending on whether we want to keep emacs-$version.exe around. It does
> not touch your build files, just the standard install process. But I
> appreciate that your builds with install-strip are way leaner now.


I think it is my build process which is causing the problem. Zip doesn't
do hard links, as far as I know. There isn't a fix for this.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Juan José García-Ripoll
>>  <juanjose.garciaripoll@gmail.com>
>> Date: Sun, 29 Mar 2020 00:36:12 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> > (This is a secondary point, but Emacs is not linked statically, at
>> > least not by default.
>> 
>> Just a side note: the build-zips.sh script that makes the Windows
>> distribution files always uses the flag -static when building emacs.
>
> I don't think the build should use that.  It is certainly not the
> default.

IIRC, without this the no-deps version fails, because of a requirement
for libwinpthreads. I think it has been there since I added those files
and it was discussed at the time. I'm happy for anyone to try taking it
out, rebuilding emacs and checking whether it is still necessary.


Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Juan José García-Ripoll <juanjose.garciaripoll@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>>> Just a side note: the build-zips.sh script that makes the Windows
>>> distribution files always uses the flag -static when building emacs.
>> I don't think the build should use that.  It is certainly not the
>> default.
>
> So where is the default? Is the default build process not the one in
> emacs/admin/nt or, in case one builds from sources without creating zip
> files, the one in emacs/nt/? I am reporting data from running those two
> procedures. If you are claiming I am not doing the standard thing, where
> is the standard thing?
>
> Apologies if I may sometimes sound rude, but I am committing my workflow
> and part of my group's to Emacs for various reasons (org-mode, latex, ebib,
> etc) and I would like to contribute make the upcoming releases work
> nicely with our systems.


It's fine to critique things, but we also need fixes that are
better. The hard link ain't got to work in a zip. But hard-linking is
right for the other situations, so changing the build (i.e. make) to fix
this isn't probably not a good idea. So, we have a minor problem, with
no obvious solution. Again, I doubt that 6Mb is a problem for anyone; is
it causing you any difficulties?

If you are really raring to make Emacs work nicely with other systems,
then the two main problems with the windows build at the moment are the
lack of an MSI, and a mechanism for signing the binaries so they don't
produce a warning on running.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: juanjose.garciaripoll@gmail.com,  emacs-devel@gnu.org
> Date: Sun, 29 Mar 2020 13:52:43 +0100
> 
> >> The pretests should be stripped actually. I use make install for
> >> snapshot builds. The ones with an actual version number are make
> >> install-strip'd.
> >
> > Given the size, I don't think they are stripped.
> 
> 
> 6.4Mb?

No, 128MB.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: Juan José García-Ripoll
>  <juanjose.garciaripoll@gmail.com>,
>   emacs-devel@gnu.org
> Date: Sun, 29 Mar 2020 13:59:40 +0100
> 
> >> Just a side note: the build-zips.sh script that makes the Windows
> >> distribution files always uses the flag -static when building emacs.
> >
> > I don't think the build should use that.  It is certainly not the
> > default.
> 
> IIRC, without this the no-deps version fails, because of a requirement
> for libwinpthreads.

We should try building Emacs so that it isn't dependent on
libwinpthreads.  (Mine isn't, but I'm using a different MinGW flavor.)

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Juan José García-Ripoll
>  <juanjose.garciaripoll@gmail.com>
> Date: Sun, 29 Mar 2020 11:38:08 +0200
> 
> So let me pause here and retrace this discussion, because I feel like
> the cat that is chasing the red pointer with you guys.
> 
> - My original question was: what is the official build process for
>   making standalone Emasc packages, so as to reproduce the content of
>   the official distros and understand what goes in. That is in a
>   separate thread.

Part of the problem is caused by using "build process" to mean 2
separate things: (a) the process of building Emacs from sources and
installing it locally via "make install", and (b) the process of
producing the binary distribution with or without the optional
libraries (a.k.a. "dependencies").  Let's call (b) something else,
like "distribution" or "building zip".

> Regarding i)
> - I have verified that the distro build process reported in admin/nt
>   starts with CFLAGS="-O2 -static" by default. Some options change it
>   to CFLAGS="-O3 -static -g", but the -static flag is never dropped.

Right, that's the libwinpthread fiasco.  It's an annoyance, but it
isn't too important, since only relatively small libraries are being
linked in statically.

> - I have verified that the files are not hard links. Actually, runing
>   "configure" with the official sources produces this in config.status
>             S["LN_S_FILEONLY"]="/bin/ln -s"
>   so there is no way this is going to create hard links.

Right, this is due to a bug that is embarrassingly old, it turns out.
I've just fixed that on master; for Emacs 27 you can work around by
saying

  make install-strip LN_S_FILEONLY=/bin/ln

when you install Emacs you've built.

>   This makes sense for msys/mingw because in general there may be
>   out there file systems which are still FAT-based and do not
>   support hard links.

Since you are preparing a special-purpose distribution (or so I
understand), you could still use hard links if your users don't need
to support FAT32 filesystems.

In any case, given the small size of the Emacs binary when stripped, I
don't think having two copies of that would be a problem that requires
us to give up on having binaries of old versions available, as on
other systems.

> >> Just a side note: the build-zips.sh script that makes the Windows
> >> distribution files always uses the flag -static when building emacs.
> > I don't think the build should use that.  It is certainly not the
> > default.
> 
> So where is the default? Is the default build process not the one in
> emacs/admin/nt or, in case one builds from sources without creating zip
> files, the one in emacs/nt/? I am reporting data from running those two
> procedures. If you are claiming I am not doing the standard thing, where
> is the standard thing?

See above: this is part of the confusion due to overloaded meaning of
"build process".

> Emacs built from sources:
> $ fsutil hardlink list emacs.exe
> \Users\juanj\emacs-27\bin\emacs.exe
> $ fsutil hardlink list emacs-27.0.90.exe
> \Users\juanj\emacs-27\bin\emacs-27.0.90.exe

Try the same in the src directory.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Angelo Graziosi]

Eli Zaretskii wrote:

> I've just fixed that on master;

Now my build on Windows via MSYS2 creates an hardlink instead then s symlink. Notice that MSYS2 by default does not enable symlink so the using 'ln -s ...' really copies the file. So I enabled Windows native symlink with  

rem To activate windows native symlinks uncomment next line
set MSYS=winsymlinks:nativestrict

in /msys2_shell.cmd

(maybe W10 needs also the developpers mode). In this way my builds had 

  1 mar 13.55 emacs-27.0.90.exe
  1 mar 14.01 emacs.exe -> emacs-27.0.90.exe


What do I have to do to get back to the way I used to be? I don't want to delete the hardlink and add the symlink manually.

BTW, why Emacs, on Windows, has 3 executable?

  runemacs.exe
  emacs-27.0.90.exe
  emacs.exe (hardlink, now)

Yes, runemacs.exe to remove the DOS window (if I remember): why this cannot be done with a single EXE? On Windows there ar a lot of apps (IDE, Editore, Browser etc) which have only one EXE to be started with (no DOS window etc). WHY not Emacs? This could be another simplification in the spirit of this discussion...

Ciao,
  Angelo.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> Date: Sun, 29 Mar 2020 19:01:50 +0200 (CEST)
> From: Angelo Graziosi <angelo.g0@libero.it>
> 
> What do I have to do to get back to the way I used to be?

  make install LN_S_FILEONLY='/bin/ln -s'

> BTW, why Emacs, on Windows, has 3 executable?
> 
>   runemacs.exe
>   emacs-27.0.90.exe
>   emacs.exe (hardlink, now)
> 
> Yes, runemacs.exe to remove the DOS window (if I remember): why this
> cannot be done with a single EXE?

Because emacs.exe is a console program, not a GUI program.

> On Windows there ar a lot of apps (IDE, Editore, Browser etc) which have only one EXE to be started with (no DOS window etc).

Those other apps are only GUI, whereas Emacs can be run both as a GUI
and a text-mode program.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: juanjose.garciaripoll@gmail.com,  emacs-devel@gnu.org
>> Date: Sun, 29 Mar 2020 13:52:43 +0100
>> 
>> >> The pretests should be stripped actually. I use make install for
>> >> snapshot builds. The ones with an actual version number are make
>> >> install-strip'd.
>> >
>> > Given the size, I don't think they are stripped.
>> 
>> 
>> 6.4Mb?
>
> No, 128MB.

emacs-27.0.90.2 size is 6Mb
emacs-28.0.50-snapshot is 128Mb

I think the stripping is working as (I) expected.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: Juan José García-Ripoll
>>  <juanjose.garciaripoll@gmail.com>,
>>   emacs-devel@gnu.org
>> Date: Sun, 29 Mar 2020 13:59:40 +0100
>> 
>> >> Just a side note: the build-zips.sh script that makes the Windows
>> >> distribution files always uses the flag -static when building emacs.
>> >
>> > I don't think the build should use that.  It is certainly not the
>> > default.
>> 
>> IIRC, without this the no-deps version fails, because of a requirement
>> for libwinpthreads.
>
> We should try building Emacs so that it isn't dependent on
> libwinpthreads.  (Mine isn't, but I'm using a different MinGW flavor.)

Well, my records tell me that I have used "-static" since Emacs-25.1
which is the first version with a 64bit build, and the first version I
built.

I cannot find any discussion about this on the mailing list other than this:

https://lists.gnu.org/archive/html/emacs-devel/2016-11/msg00271.html

which just shows my failing memory and a different problem.

Anyway, I just tried a build without it on an Emacs-28 snapshot. Emacs
fails to launch with an "libgmp-10.dll is missing" error.

I am happy if someone wants to work out how to remove "-static" from the
build files, but I have neither the time nor expertise.

Phil

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: juanjose.garciaripoll@gmail.com,  emacs-devel@gnu.org
> Date: Sun, 29 Mar 2020 19:18:49 +0100
> 
> Anyway, I just tried a build without it on an Emacs-28 snapshot. Emacs
> fails to launch with an "libgmp-10.dll is missing" error.

That's expected: we now support bignums, and that requires GMP.  It
simply means libgmp-10.dll should be part of the basic (no-deps)
package.

Re: Avoid duplicate emacs.exe / emacs-$version.exe

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: juanjose.garciaripoll@gmail.com,  emacs-devel@gnu.org
>> Date: Sun, 29 Mar 2020 19:18:49 +0100
>> 
>> Anyway, I just tried a build without it on an Emacs-28 snapshot. Emacs
>> fails to launch with an "libgmp-10.dll is missing" error.
>
> That's expected: we now support bignums, and that requires GMP.  It
> simply means libgmp-10.dll should be part of the basic (no-deps)
> package.


If I copy libgmp-10.dll alongside emacs.exe, as with libXpm-noX4, then
indeed it works. So may be we don't need "-static" any more. It's a long
time since it was tested as you can see. The source for libgmp is in the
source dump.

Another option would be to deprecate -no-deps or remove it
wholesale, Or, alternatively, to not guarantee that it runs (so, I'd
remove libXpm-noX4 also).

Thoughts? Presume this would be on master cause it's a bit late for
Emacs-27.

Phil