bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Lars Ingebrigtsen]

I wonder whether we should AOT a few more files by default.  I think we
should probably aim for a typical user not seeing compilation firing off
on normal usage, but what's "normal usage" anyway?

In any case, "emacs -Q" and then `C-x C-b' gives me this:

Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-lib.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-extra.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/help-mode.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/gv.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-macs.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-seq.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/rx.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/subr-x.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/icons.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/warnings.el...
Compiling /home/larsi/src/emacs/nativecomp/lisp/display-line-numbers.el...
Compilation finished.

This suggests to me that we should AOT a bunch of these libraries
(rx/subr-x/etc) by default that are virtually impossible to use Emacs
without.

Any opinions?


In GNU Emacs 29.0.50 (build 162, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2022-10-16 built on downe
Repository revision: 45aabe6edae8d841a8985853394baddad4a1949e
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Ubuntu 22.04.1 LTS

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Andrea Corallo]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I wonder whether we should AOT a few more files by default.  I think we
> should probably aim for a typical user not seeing compilation firing off
> on normal usage, but what's "normal usage" anyway?
>
> In any case, "emacs -Q" and then `C-x C-b' gives me this:
>
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-lib.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-extra.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/help-mode.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/gv.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-macs.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-seq.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/rx.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/subr-x.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/icons.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/warnings.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/display-line-numbers.el...
> Compilation finished.
>
> This suggests to me that we should AOT a bunch of these libraries
> (rx/subr-x/etc) by default that are virtually impossible to use Emacs
> without.
>
> Any opinions?

I think as well we should complile AOT them.

  Andrea

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Eli Zaretskii]

> Cc: Andrea Corallo <akrl@sdf.org>
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 17 Oct 2022 11:11:22 +0200
> 
> I wonder whether we should AOT a few more files by default.  I think we
> should probably aim for a typical user not seeing compilation firing off
> on normal usage, but what's "normal usage" anyway?

Indeed, what is "normal usage"?

> In any case, "emacs -Q" and then `C-x C-b' gives me this:
> 
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-lib.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-extra.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/help-mode.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/gv.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-macs.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/cl-seq.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/rx.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/subr-x.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/icons.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/emacs-lisp/warnings.el...
> Compiling /home/larsi/src/emacs/nativecomp/lisp/display-line-numbers.el...
> Compilation finished.
> 
> This suggests to me that we should AOT a bunch of these libraries
> (rx/subr-x/etc) by default that are virtually impossible to use Emacs
> without.

Try with -nw, and you will see more.  is that also "normal usage"?

In Emacs 28 we added several files to AOT for that reason, but AFAIR
they were removed on master.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> This suggests to me that we should AOT a bunch of these libraries
>> (rx/subr-x/etc) by default that are virtually impossible to use Emacs
>> without.
>
> Try with -nw, and you will see more.  is that also "normal usage"?

It was pretty much the same list, and yes.

> In Emacs 28 we added several files to AOT for that reason, but AFAIR
> they were removed on master.

It's obviously a matter of taste -- we don't want to add libraries
needlessly, but there's some (non-pre-loaded) libraries (I'd guesstimate
about a dozen) that virtually all users will end up loading, so AOT-ing
them makes sense to me.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> In Emacs 28 we added several files to AOT for that reason, but AFAIR
> they were removed on master.

Looking at Makefile.in, it seems like they were removed by the following
change?

Which looks like a good change -- they don't need to be compiled
first -- but it's not like we stopped eln-ing (for instance) subr-x on
purpose.

But I guess we'd have to introduce a new mechanism for this, like
COMPILE_ELN_EXTRA or something, if we want to AOT them anyway?

commit 99c276b3c0aee5599c1d281c1e9fe223810784ca
Author:     Andrea Corallo <akrl@sdf.org>
AuthorDate: Tue Nov 30 13:49:36 2021 +0100
Commit:     Andrea Corallo <akrl@sdf.org>
CommitDate: Tue Nov 30 15:42:41 2021 +0100

    Revert "Preload paren.el"
    
    Reverting as the previous commit make this fix not anymore necessary.
    
    This reverts commit 340e527bed83ff0382446132c871088ad61d1745.

diff --git a/lisp/Makefile.in b/lisp/Makefile.in
--- a/lisp/Makefile.in
+++ b/lisp/Makefile.in
@@ -88,21 +88,10 @@
 COMPILE_FIRST = \
 	$(lisp)/emacs-lisp/macroexp.elc \
 	$(lisp)/emacs-lisp/cconv.elc    \
 	$(lisp)/emacs-lisp/byte-opt.elc \
 	$(lisp)/emacs-lisp/bytecomp.elc
 ifeq ($(HAVE_NATIVE_COMP),yes)
-COMPILE_FIRST += \
-	$(lisp)/emacs-lisp/comp.elc \
-	$(lisp)/emacs-lisp/comp-cstr.elc \
-	$(lisp)/emacs-lisp/cl-macs.elc \
-	$(lisp)/emacs-lisp/rx.elc \
-	$(lisp)/emacs-lisp/cl-seq.elc \
-	$(lisp)/help-mode.elc \
-	$(lisp)/emacs-lisp/cl-extra.elc \
-	$(lisp)/emacs-lisp/gv.elc \
-	$(lisp)/emacs-lisp/seq.elc \
-	$(lisp)/emacs-lisp/cl-lib.elc \
-	$(lisp)/emacs-lisp/warnings.elc \
-	$(lisp)/emacs-lisp/subr-x.elc

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 58580@debbugs.gnu.org,  akrl@sdf.org
> Date: Mon, 17 Oct 2022 13:25:51 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> This suggests to me that we should AOT a bunch of these libraries
> >> (rx/subr-x/etc) by default that are virtually impossible to use Emacs
> >> without.
> >
> > Try with -nw, and you will see more.  is that also "normal usage"?
> 
> It was pretty much the same list, and yes.
> 
> > In Emacs 28 we added several files to AOT for that reason, but AFAIR
> > they were removed on master.
> 
> It's obviously a matter of taste -- we don't want to add libraries
> needlessly, but there's some (non-pre-loaded) libraries (I'd guesstimate
> about a dozen) that virtually all users will end up loading, so AOT-ing
> them makes sense to me.

So maybe we should reinstate the ones we pre-compiled in Emacs 28.
FTR, those were:

 autoload
 byte-opt
 bytecomp
 cconv
 charscript
 cl-extra
 cl-lib
 cl-macs
 cl-seq
 comp
 comp-cstr
 emoji-zwj
 gv
 help-mode
 rx
 seq
 subr-x
 warnings

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: akrl@sdf.org,  58580@debbugs.gnu.org
> Date: Mon, 17 Oct 2022 14:13:02 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > In Emacs 28 we added several files to AOT for that reason, but AFAIR
> > they were removed on master.
> 
> Looking at Makefile.in, it seems like they were removed by the following
> change?

Andrea removed them because their addition in the final stages of
Emacs 28.1's pretest was in response to last-minute problems at
startup in some configurations, in particular when Emacs was invoked
with -nw.  Those problems were lately fixed on master in another way,
so Andrea didn't want to keep that semi-kludgey solution which became
unnecessary.

> Which looks like a good change -- they don't need to be compiled
> first -- but it's not like we stopped eln-ing (for instance) subr-x on
> purpose.

See above.

However, now we are saying that having some files precompiled is more
convenient, as it makes the first startup of a newly built/installed
Emacs faster and smoother.  (It only affects the first startup after
building a new version.)  So the motivation is different, but I don't
see why the same solution couldn't satisfy this one as well.  After
all, the solution is solid and tested by 2 releases.

> But I guess we'd have to introduce a new mechanism for this, like
> COMPILE_ELN_EXTRA or something, if we want to AOT them anyway?

Why not use the same mechanism?

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Eli Zaretskii]

> Cc: akrl@sdf.org, 58580@debbugs.gnu.org
> Date: Mon, 17 Oct 2022 16:34:05 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
>  autoload
>  byte-opt
>  bytecomp
>  cconv
>  charscript
>  cl-extra
>  cl-lib
>  cl-macs
>  cl-seq
>  comp
>  comp-cstr
>  emoji-zwj
>  gv
>  help-mode
>  rx
>  seq
>  subr-x
>  warnings

Btw, with Emacs 29 I see the following compiled on the first -nw
startup of every new build:

 cl-lib
 cl-loaddefs
 cl-seq
 rx
 subr-x
 icons
 warnings
 display-line-numbers

And I'm hard-pressed to explain why icons and display-line-numbers are
there.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

[proposed list snipped, but sounds good to me]

>  warnings
>  display-line-numbers
>
> And I'm hard-pressed to explain why icons and display-line-numbers are
> there.

icons because of warnings, but I don't know why display-line-numbers.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> However, now we are saying that having some files precompiled is more
> convenient, as it makes the first startup of a newly built/installed
> Emacs faster and smoother.  (It only affects the first startup after
> building a new version.)  So the motivation is different, but I don't
> see why the same solution couldn't satisfy this one as well.  After
> all, the solution is solid and tested by 2 releases.

I'm not 100% sure I understand -- by "solution" here, you mean
COMPILE_FIRST?

>> But I guess we'd have to introduce a new mechanism for this, like
>> COMPILE_ELN_EXTRA or something, if we want to AOT them anyway?
>
> Why not use the same mechanism?

Because it's slower to do eln compilation in COMPILE_FIRST -- it's
faster later, once Emacs has been built with nativecomp.  So these extra
eln bits should be done around the same time as compile-main, or rather:
During compile-main, so that we get max parallelism, too.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Andrea Corallo]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> However, now we are saying that having some files precompiled is more
>> convenient, as it makes the first startup of a newly built/installed
>> Emacs faster and smoother.  (It only affects the first startup after
>> building a new version.)  So the motivation is different, but I don't
>> see why the same solution couldn't satisfy this one as well.  After
>> all, the solution is solid and tested by 2 releases.
>
> I'm not 100% sure I understand -- by "solution" here, you mean
> COMPILE_FIRST?

The quick fix on 28 was to compile AOT those files, using jit
compilation was causing as Eli mentioned some circular dependency
issues.

The fix that solves those issues is
9b381a95ef6cd9194d64bfb17fd50bb99fa6cd32 (and this is only on master).
In 29 we are free to compile AOT those files or not, the infrastructure
was made robust to handle that.

  Andrea

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> However, now we are saying that having some files precompiled is more
> convenient, as it makes the first startup of a newly built/installed
> Emacs faster and smoother.  (It only affects the first startup after
> building a new version.)

After thinking about this a bit more, I think this is a pretty
compelling argument: Multi-user machines basically don't exist any more,
so there's little to be gained by having the default build do the AOT
for these additional files in the development build.

And distributions will do a full AOT distribution anyway, or will do a
distribution without any .eln file distributed -- it's hard to envision
a distribution doing a distribution with just a handful of .eln files.

So this would just added maintenance burden and further complicate the
build without any gains for users, and I'm therefore closing this bug
report.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: akrl@sdf.org,  58580@debbugs.gnu.org
> Date: Mon, 17 Oct 2022 21:35:31 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > However, now we are saying that having some files precompiled is more
> > convenient, as it makes the first startup of a newly built/installed
> > Emacs faster and smoother.  (It only affects the first startup after
> > building a new version.)  So the motivation is different, but I don't
> > see why the same solution couldn't satisfy this one as well.  After
> > all, the solution is solid and tested by 2 releases.
> 
> I'm not 100% sure I understand -- by "solution" here, you mean
> COMPILE_FIRST?

No, I mean "make $(elnlisp)" in src/Makefile.in.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> No, I mean "make $(elnlisp)" in src/Makefile.in.

Right.

bug#58580: 29.0.50; Ahead-of-time compilation for a few more files?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: akrl@sdf.org,  58580@debbugs.gnu.org
> Date: Mon, 17 Oct 2022 21:31:29 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> [proposed list snipped, but sounds good to me]
> 
> >  warnings
> >  display-line-numbers
> >
> > And I'm hard-pressed to explain why icons and display-line-numbers are
> > there.
> 
> icons because of warnings, but I don't know why display-line-numbers.

I know why display-line-numbers is compiled: it happens whenever you
invoke some command that needs tabulated-list-mode.  For me, it's
usually list-processes (which I invoke to see if Emacs is running a
native compilation).