Re: Loading tramp for dump goes into infinite regress

[Lynn Winebarger <owinebar@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3530 bytes --]

On Sun, Jul 24, 2022 at 12:31 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Lynn Winebarger <owinebar@gmail.com>
> > Date: Sun, 24 Jul 2022 12:20:47 -0400
> > Cc: michael.albinus@gmx.de, emacs-devel@gnu.org
> >
> > it seems like something undesirable is going on between dumping and
> > the native compilation unit.
>
> I find this hard to believe.

If I can recreate it in a more direct way on my personal machines, I'll put
in a bug with details including the required modifications to tramp.el (to
remove the problematic require of tramp-loaddefs).
I also observed the same problem/fix with:
  gnus/message
  progmodes/gdb-mi
  calc/calc-ext
  gnus/gnus-sum
  gnus/gnus-art
More troublesome, though is that emacs segfaulted when it attempted to load
nxml/rng-pttrn, whether loading native-compiled, byte-compiled, or source.

Otherwise, I was able to resolve the various issues (including some
extensive surgery on vc/ediff-X files to stamp out the circular loading).
I am going to have to do a real bootstrap to avoid the "incoherent eln"
error, though, and somehow force native compilation for all the libraries
loaded in the dump (that are not on the compiler's "forbidden" list,
anyway).

> Btw, you are aware that dumping *.eln files basically dumps just their
> names, not their code?  And that, when you start Emacs after dumping,
> it will load all of those *.eln files one by one, which takes time
> (*.eln files are just shared libraries, like *.so files), and use up
> shared-library and handle slots of the Emacs process?  So I'm not even
> sure doing this would make sense from the performance POV: it could be
> that startup will slower, not faster.

I am aware that dumping the eln files produces an indirection to the shared
library, but not the details of the implementation.
I would expect (or at least hope) that the effect of the evaluation that is
done on loading to be stored in the dump and for the shared libraries to
get loaded into the process without repeating the evaluation.  So
things like the order of customization groups should be fixed.
I would also expect that whatever the effect is on the emacs process with
.eln files loaded by the dumped file, it is no worse than it would be if
loaded as part of my .emacs file.
Another benefit I expect from native-compilation, dumped or not, is more
efficient memory use when running multiple emacs processes.  With dumping,
I would expect (or hope for) better garbage collector behavior since the
amount of allocation required for the loaded modules should be
pre-determined (whether byte- or native-compiled).  If the image is 300MB
(including the shared libraries), so be it, as long as the memory is shared
between multiple processes.

I'd also like a baseline expectation of performance with native-compiled
libraries in the dumped image.  To the extent that dumping verifies that
there is no cyclic loading, I can always try the approach to linking I
inquired about on this list about a month ago: just catenating the source
files together with some minor alterations.  I'm not sure how large a
source file the native compiler (or libgccjit) will handle, but it would be
interesting to see.

> In any case, I'd suggest to get this working with *.elc files in a
> build without native compilation support, before you try it with
> native compilation.

First I'll see if I can get the ordering straightened out with just the
core emacs files required by these packages, then do a test between
byte-compiled and native-compiled on those.

Lynn

[-- Attachment #2: Type: text/html, Size: 4414 bytes --]