Working fully native-compiled "mega dump" (was Re: Loading tramp for dump goes into infinite regress)

[Lynn Winebarger <owinebar@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3328 bytes --]

On Sat, Aug 6, 2022, 11:39 AM Lynn Winebarger <owinebar@gmail.com> wrote:

> On Sat, Aug 6, 2022, 8:57 AM Lynn Winebarger <owinebar@gmail.com> wrote:
>
>> On Sat, Aug 6, 2022, 2:07 AM Lynn Winebarger <owinebar@gmail.com> wrote:
>>
>>> I've added another flag variable that allows me to turn off the ban on
>>> implicit loading in dump mode.
>>>
>>
>> The dump with all ~2500 ELN libraries loaded completed last night.
>> Unfortunately during startup (before any frame opens) one of them is
>> evaluating an expression with an argument that should never produce nil but
>> somehow is nil.
>> Fortunately I can now redump with just the offending library in
>> site-load.  The error prints fairly quickly when run normally, but shared
>> libraries evidently open a lot slower under GDB.
>> Hopefully I'll get the same error with the smaller dump and be able to
>> determine if there's an issue in the dump file, the dump loading, or
>> something else.
>>
>
> It started up fine when I only explicitly added the one library with the
> failing expression, and then again when I dumped after loading the original
> list up to that library.
> The good news is that emacs starts in a couple of seconds with 1198 ELN
> files in the dump.  I haven't been able to compile with profiling enabled
> on this system, but I can tell you that is a couple of orders of magnitude
> faster than just loading them in .init.
>

The bug seems to be from a couple of older (2010-2012) library files that
are dynamically scoped and use copy-face with the "frame" parameter.  All
the other combinations I've tried start up without a problem, so I'm just
going to leave these packages out of the dump.  Also excluded are most
lsp-mode libraries, because they seem to slow down the startup process
(like 4 seconds instead of 2 or less}, vterm and zmq (dynamic modules can't
be dumped), and a few modes that insist on attempting to download things on
startup.
What's included:
All of CEDET
All files in emacs-lisp
Most if not all files in tramp/mail/net
Everything in progmode that doesn't depend on an external utility (namely
moz) being installed
Any TeX or reference/citation functionality (reftex, bibtex, bibliography,
citar, citeproc)
Icon/emoji/image support (e.g alltheicons, major-mode-icons)
All completion and widening/narrowing libraries - helm, ivy, counsel,
company, auto-complete,
Perspective, window-purpose, and similar
Evil packages
Treemacs
Many other miscellaneous packages I'm forgetting.

All in the dump file is about 184 MB for an older Linux on an x86_64
architecture.  That would include all the lsp-mode packages from Melpa, or
at least the ones with more than a nominal number of downloads.  There are
2000-2500 ELN files that will be loaded.  When I loaded these from my init
file I could make a cup of coffee, maybe do a crossword before it would be
ready to use.

One of the packages with the dynamic scoping/face issues is tabbar-ruler,
which I only started using during this exercise and think is fantastic.
Hopefully it will work when required from the init file, as it does
currently.

At any rate it's fantastic at startup.  The customization interface is very
responsive without the usual lag from lazy construction of the
customization groups.  I'll have to see how it behaves after it's been
running a few days.

Lynn

[-- Attachment #2: Type: text/html, Size: 5579 bytes --]