emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[Gregor Zattler]

Hello,

among several runs of

time /usr/bin/emacs-gtk -Q -nw -f save-buffers-kill-emacs

which starts emacs 27.1 as precompiled in debian / Bullseye
and kills it again, a low time values on my laptop are e.g.:

real    0m0,328s
user    0m0,262s
sys     0m0,058s


while for emacs v29.0 the same command line achives low
values like this one:

real    0m1,007s
user    0m0,847s
sys     0m0,154s

The 29.0 version is compiled with native-compilation
support.  Does this explain the differences?


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[Eli Zaretskii]

> From: Gregor Zattler <telegraph@gmx.net>
> Date: Mon, 23 Jan 2023 15:08:15 +0100
> 
> time /usr/bin/emacs-gtk -Q -nw -f save-buffers-kill-emacs
> 
> which starts emacs 27.1 as precompiled in debian / Bullseye
> and kills it again, a low time values on my laptop are e.g.:
> 
> real    0m0,328s
> user    0m0,262s
> sys     0m0,058s
> 
> 
> while for emacs v29.0 the same command line achives low
> values like this one:
> 
> real    0m1,007s
> user    0m0,847s
> sys     0m0,154s
> 
> The 29.0 version is compiled with native-compilation
> support.  Does this explain the differences?

It could.  Emacs with native-compilation needs to load all the
preloaded Lisp packages as shared libraries, as opposed to just
mapping the pdumper file into the address space in Emacs 27.  To be
sure, build Emacs 29 without native-compilation, and then compare.

Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[Gregor Zattler]

Hi Eli, emacs users,
* Eli Zaretskii <eliz@gnu.org> [2023-01-23; 17:08 +02]:
>> From: Gregor Zattler <telegraph@gmx.net>
>> Date: Mon, 23 Jan 2023 15:08:15 +0100
>>
>> time /usr/bin/emacs-gtk -Q -nw -f save-buffers-kill-emacs
>>
>> which starts emacs 27.1 as precompiled in debian / Bullseye
>> and kills it again, a low time values on my laptop are e.g.:
>>
>> real    0m0,328s
>> user    0m0,262s
>> sys     0m0,058s
>>
>>
>> while for emacs v29.0 the same command line achives low
>> values like this one:
>>
>> real    0m1,007s
>> user    0m0,847s
>> sys     0m0,154s
>>
>> The 29.0 version is compiled with native-compilation
>> support.  Does this explain the differences?
>
> It could.  Emacs with native-compilation needs to load all the
> preloaded Lisp packages as shared libraries, as opposed to just
> mapping the pdumper file into the address space in Emacs 27.  To be
> sure, build Emacs 29 without native-compilation, and then compare.

So I did.  The lowest values for the same command with a
slightly newer Version which was built with --with-native-compilation=no
are:


$ time src/emacs -Q -nw -f save-buffers-kill-emacs

real    0m0,596s
user    0m0,483s
sys     0m0,105s


while the very same tree build  with --with-native-compilation

$ time src/emacs -Q -nw -f save-buffers-kill-emacs

real    0m1,082s
user    0m0,902s
sys     0m0,169s

This shows: a) native compilation slows Emacs' startup (and
shutdown?).  b) Without native compilation support Emacs
27.1 is faster in this nano-benchmark than Emacs 29.0.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[John Yates]

I work on a product whose structure involves literally hundreds of
DLLs / SOs.  We tolerate slow start-up in-house because it optimizes
the compile / edit / debug cycle.

Packaging the final product involves an extra step to collapse our
hundreds of images into just a handful via a secondary link step.
I wonder if there is anything analogous that Emacs could do.

/john

Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[Eli Zaretskii]

> From: John Yates <john@yates-sheets.org>
> Date: Thu, 26 Jan 2023 20:16:36 -0500
> 
> I work on a product whose structure involves literally hundreds of
> DLLs / SOs.  We tolerate slow start-up in-house because it optimizes
> the compile / edit / debug cycle.

How many hundreds?  Emacs 29 loads almost 300 *.eln files at startup.

> Packaging the final product involves an extra step to collapse our
> hundreds of images into just a handful via a secondary link step.
> I wonder if there is anything analogous that Emacs could do.

I'm not sure this is possible with loading natively-compiled Lisp in
Emacs, because there are some unique conditions and requirements in
our case, like the need to be able to run both installed and
un-installed Emacs, find the correct *.eln files on the system, etc.
It isn't like loading a shared library from a system-wide directory
where shared libraries live.

In any case, please start a discussion about this on emacs-devel, and
please CC Andrea Corallo <akrl@sdf.org>, who developed the native
compilation and tried several alternatives for dumping Emacs with
natively-compiled Lisp before arriving at what we do today.  It's a
complex problem with many facets.

Fwd: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]

Reposting to emacs-devel at Eli's behest.

Re my suggested parallel, Eli asked:

> How many hundreds?  Emacs 29 loads almost 300 *.eln files at startup.

I have never needed to answer that question.

I work on the code generation middle-end.  When I modify one of its
fundamental headers over 350 modules get rebuilt.  (That number
reflects poor dependency management, which is being addressed.)
There are probably just as many modules unrelated to code generation
(e.g. UI, graphics, etc).

/john
---------- Forwarded message ---------
From: John Yates <john@yates-sheets.org>
Date: Thu, Jan 26, 2023 at 8:16 PM
Subject: Re: emacs empty startup time increased from v 27.1 to 29.0 by
factor 3?
To: <help-gnu-emacs@gnu.org>


I work on a product whose structure involves literally hundreds of
DLLs / SOs.  We tolerate slow start-up in-house because it optimizes
the compile / edit / debug cycle.

Packaging the final product involves an extra step to collapse our
hundreds of images into just a handful via a secondary link step.
I wonder if there is anything analogous that Emacs could do.

[-- Attachment #2: Type: text/html, Size: 2416 bytes --]

Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 2621 bytes --]

Hi, John,
The closest thing in Emacs to what you describe is dumping, which currently
isn't supported for native compilation, unless something has changed that
I'm not aware of.
Last summer I did an experiment and modified 28.1 to support dumping native
compilation.  I had a startup that was loading all libraries included with
emacs and approximately an additional 1000 packages installed in site-lisp
(lots of manual modifications to make that happen).
Loading those packages at startup as regular packages, after ensuring all
were compiled ahead of time, increased startup time to 15 minutes or more.
The dumped version only took 5 seconds or so.
I discarded that experiment, but have done a similar site-lisp installation
with maybe half of those additional packages.  The dump produced by
"dump-emacs-portable" starts almost instantly.  Even without dumping,
startup time is only 30 seconds or so.  For that matter, it takes 12
minutes to dump the byte-compiled function, where dumping the natively
compiled version would take over 30 minutes.  It's not a completely fair
comparison since (if memory serves) the natively compiled version loaded
4000-5000 files where this version is only loading ~2200 libraries.

Loading natively compiled libraries does do some additional checking to
compare source file to the compiled library by checksum.  Maybe that's part
of the 3x you're seeing?

Lynn


On Fri, Jan 27, 2023, 12:22 PM John Yates <john@yates-sheets.org> wrote:

> Reposting to emacs-devel at Eli's behest.
>
> Re my suggested parallel, Eli asked:
>
> > How many hundreds?  Emacs 29 loads almost 300 *.eln files at startup.
>
> I have never needed to answer that question.
>
> I work on the code generation middle-end.  When I modify one of its
> fundamental headers over 350 modules get rebuilt.  (That number
> reflects poor dependency management, which is being addressed.)
> There are probably just as many modules unrelated to code generation
> (e.g. UI, graphics, etc).
>
> /john
> ---------- Forwarded message ---------
> From: John Yates <john@yates-sheets.org>
> Date: Thu, Jan 26, 2023 at 8:16 PM
> Subject: Re: emacs empty startup time increased from v 27.1 to 29.0 by
> factor 3?
> To: <help-gnu-emacs@gnu.org>
>
>
> I work on a product whose structure involves literally hundreds of
> DLLs / SOs.  We tolerate slow start-up in-house because it optimizes
> the compile / edit / debug cycle.
>
> Packaging the final product involves an extra step to collapse our
> hundreds of images into just a handful via a secondary link step.
> I wonder if there is anything analogous that Emacs could do.
>
>

[-- Attachment #2: Type: text/html, Size: 4457 bytes --]

Re: Fwd: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?

[Eli Zaretskii]

> From: John Yates <john@yates-sheets.org>
> Date: Fri, 27 Jan 2023 12:20:57 -0500
> Cc: Andrea Corallo <akrl@sdf.org>
> 
> Re my suggested parallel, Eli asked:
> 
> > How many hundreds?  Emacs 29 loads almost 300 *.eln files at startup.
> 
> I have never needed to answer that question.
> 
> I work on the code generation middle-end.  When I modify one of its
> fundamental headers over 350 modules get rebuilt.  (That number
> reflects poor dependency management, which is being addressed.)
> There are probably just as many modules unrelated to code generation
> (e.g. UI, graphics, etc).
> 
> /john
> ---------- Forwarded message ---------
> From: John Yates <john@yates-sheets.org>
> Date: Thu, Jan 26, 2023 at 8:16 PM
> Subject: Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?
> To: <help-gnu-emacs@gnu.org>
> 
> I work on a product whose structure involves literally hundreds of
> DLLs / SOs.  We tolerate slow start-up in-house because it optimizes
> the compile / edit / debug cycle.
> 
> Packaging the final product involves an extra step to collapse our
> hundreds of images into just a handful via a secondary link step.
> I wonder if there is anything analogous that Emacs could do.

How exactly do you "collapse the hundreds of shared libraries into
just a handful by a secondary link step"?