Re: Docstring hack

[Lynn Winebarger <owinebar@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3496 bytes --]

On Sun, Jul 31, 2022, 4:03 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > OK, but I still lack some glue to understand the issue.  Specifically:
> >
> >   . the OP said "strings that are erroneously treated as docstrings in
> >     dump mode" -- where's the code which makes that mistake, and how
> >     is read_literal_string related to that mistake?
> >   . why isn't there an alternative to fix read_literal_string not to
> >     generate zero instead of the format template? the other

>     alternatives all look like partial kludges to me

The first 4 yes, but the fifth one was to remove that and just modify the
evaluator to not set those properties during dump mode.
But if the only need for this feature is in loading loaddefs.el in dump
mode, then that is overkill.  I don't understand why the docstrings are
even being extracted for the byte compiled files at all, since they would
be lazy loaded anyway.  Then you could remove the files from lisp.mk from
the dependencies of DOC in the Makefile and just leave loaddefs and the
docstrings from C source files.

In `read_literal_string` there is a hack that dates back to Emacs's
> early life where we drop the string we just read and return the
> 0 literal instead.  We do that for those strings we think are docstrings
> that will be found in etc/DOC and will be re-provided later when we call
> `Snarf-documentation` (which should then replace those 0 literals with
> appropriate integers pointing into etc/DOC).
>
> The reason for this hack is to avoid allocating the string in the heap
> (or worse, the purespace) since it's to be found lazily in
> etc/DOC instead.
>
> But we don't have a sure-fire way to recognize those strings, so we use
> a convention that they start with "double-quote backslash newline" (this
> same convention is then used in `make-docfile` in order to find those
> strings).  But some non-preloaded files also use "double-quote backslash
> newline" for other reasons, such as in `eieio-defclass-autoload`.
>

They are everywhere in fact, not just as arguments to format.


> Not sure why it's a problem for Lynn, tho: he should not try to preload
> `eieio-core.el` but only `eieio-core.elc` where the problem should not
> appear any more.


This is the conundrum of trying to do anything significant in site-load
without Makefile support.  If you're bootstrapping, then none of those
files are compiled.  So I put in a check to only load after the bootstrap
during the dump. But nothing is byte compiled at this point other than the
files in loadup.  Trying to do the byte compile from within site-load after
the bootstrap but during the dump requires pre loading the source of all
dependencies, because require and autoload hit the panic button in dump
mode.

I finally just wrote a shell script yesterday that I could invoke from
site-load to add the compiled versions of the libraries loaded in site-load
to lisp.mk (so they will get added to the list of preloaded libraries in
the final dump), then invoke make on them.  This can cause some issues
since a significant number of them require loaddefs to be loaded.  I might
be better off just adding the extra libraries directly to the environment
variable the compiler references (to determine preloaded libraries).  I
don't know if that will make a difference in determining coherence for
dumping during native-compilation.  In any case, when the shell script
calls make, it needs to arrange for the byte compiler to load loaddefs.

Lynn

[-- Attachment #2: Type: text/html, Size: 4803 bytes --]