Objects that can't be purified during dumping

Objects that can't be purified during dumping

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 539 bytes --]

Hi,
In my quest to have a "fully loaded" emacs, I'm attempting to dump a
version that loads about 2400 features in site-load.el.
Attempting to load email/rfc2047 produces an error to the effect that
purecopy does not know how to purify a char table (specifically a syntax
table).
Is there a reason objects that aren't specifically recognized as being
"purifiable" can't just be marked as pinned like hash table without the
purecopy attribute?  I mean, if I put that in the code, are there
foreseeable bugs that would result?

Thanks,
Lynn

[-- Attachment #2: Type: text/html, Size: 755 bytes --]

Re: Objects that can't be purified during dumping

[Eli Zaretskii]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Sat, 23 Jul 2022 08:19:35 -0400
> 
> In my quest to have a "fully loaded" emacs, I'm attempting to dump a version that loads about 2400 features
> in site-load.el.

Dump how? with the portable dumping?  If so, we don't yet fully
support re-dumping, although it's a goal we'd like to reach.

> Attempting to load email/rfc2047 produces an error to the effect that purecopy does not know how to purify a
> char table (specifically a syntax table).  
> Is there a reason objects that aren't specifically recognized as being "purifiable" can't just be marked as
> pinned like hash table without the purecopy attribute?  I mean, if I put that in the code, are there foreseeable
> bugs that would result?

Pure space will go away at some point, so we are unlikely to invest
any effort into fixing this.

Re: Objects that can't be purified during dumping

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]

On Sat, Jul 23, 2022, 9:27 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Lynn Winebarger <owinebar@gmail.com>
> > Date: Sat, 23 Jul 2022 08:19:35 -0400
> >
> > In my quest to have a "fully loaded" emacs, I'm attempting to dump a
> version that loads about 2400 features
> > in site-load.el.
>
> Dump how? with the portable dumping?  If so, we don't yet fully
> support re-dumping, although it's a goal we'd like to reach.
>

I might be confused, but I thought redumping referred to dumping using an
already dumped emacs.  I'm trying to bootstrap emacs with these additional
features loaded in the initial dump.

> Attempting to load email/rfc2047 produces an error to the effect that
> purecopy does not know how to purify a
> > char table (specifically a syntax table).
> > Is there a reason objects that aren't specifically recognized as being
> "purifiable" can't just be marked as
> > pinned like hash table without the purecopy attribute?  I mean, if I put
> that in the code, are there foreseeable
> > bugs that would result?
>
> Pure space will go away at some point, so we are unlikely to invest
> any effort into fixing this.
>

I recalled seeing an exchange to that effect on this list.   I'll try the
hack (on alloc.c) and see if it works as an interim solution.


Lynn

[-- Attachment #2: Type: text/html, Size: 2308 bytes --]

Re: Objects that can't be purified during dumping

[Eli Zaretskii]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Sat, 23 Jul 2022 09:59:18 -0400
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
>  Dump how? with the portable dumping?  If so, we don't yet fully
>  support re-dumping, although it's a goal we'd like to reach.
> 
> I might be confused, but I thought redumping referred to dumping using an already dumped emacs.  I'm
> trying to bootstrap emacs with these additional features loaded in the initial dump.

Ah, then sorry for my misunderstanding.

Re: Objects that can't be purified during dumping

[Stefan Monnier]

> I recalled seeing an exchange to that effect on this list.   I'll try the
> hack (on alloc.c) and see if it works as an interim solution.

I think you can just do:

    diff --git a/src/alloc.c b/src/alloc.c
    index 6e166d00d5b..acf558f3c7a 100644
    --- a/src/alloc.c
    +++ b/src/alloc.c
    @@ -5611,7 +5611,8 @@ DEFUN ("purecopy", Fpurecopy, Spurecopy, 1, 1, 0,
     static Lisp_Object
     purecopy (Lisp_Object obj)
     {
    -  if (FIXNUMP (obj)
    +  if (true
    +      || FIXNUMP (obj)
           || (! SYMBOLP (obj) && PURE_P (XPNTR (obj)))
           || SUBRP (obj))
         return obj;    /* Already pure.  */

But, the "pinned" approach as is done for hash-tables should work just
fine as well.


        Stefan

Re: Objects that can't be purified during dumping

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

On Sat, Jul 23, 2022, 11:55 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > I recalled seeing an exchange to that effect on this list.   I'll try the
> > hack (on alloc.c) and see if it works as an interim solution.
>
> I think you can just do:
>
>     diff --git a/src/alloc.c b/src/alloc.c
>     index 6e166d00d5b..acf558f3c7a 100644
>     --- a/src/alloc.c
>     +++ b/src/alloc.c
>     @@ -5611,7 +5611,8 @@ DEFUN ("purecopy", Fpurecopy, Spurecopy, 1, 1, 0,
>      static Lisp_Object
>      purecopy (Lisp_Object obj)
>      {
>     -  if (FIXNUMP (obj)
>     +  if (true
>     +      || FIXNUMP (obj)
>            || (! SYMBOLP (obj) && PURE_P (XPNTR (obj)))
>            || SUBRP (obj))
>          return obj;    /* Already pure.  */
>
> But, the "pinned" approach as is done for hash-tables should work just
> fine as well.
>

I had an off-list response indicating the purify routine is only being
called because the ELISP source uses defconst instead of defvar when
defining the symbol.  I just changed that statement to a defvar and the
problem went away. That's the most expedient solution at the moment, as
changing the c source would require re-native-compiling all the elisp files
I'm including in the dump, which takes hours to do sequentially.
I will keep this approach in my back pocket for now. Once I know the dump
will work, I may revisit the issue.

Thanks,
Lynn

[-- Attachment #2: Type: text/html, Size: 2161 bytes --]

Re: Objects that can't be purified during dumping

[Stefan Monnier]

> changing the c source would require re-native-compiling all the elisp files
> I'm including in the dump,

I thought that was not supposed to be the case (the re-compiling should
only be needed for *some* changes, tho I'm not exactly clear on what
those are).


        Stefan

Re: Objects that can't be purified during dumping

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel <emacs-devel@gnu.org>
> Date: Sun, 24 Jul 2022 11:07:05 -0400
> 
> > changing the c source would require re-native-compiling all the elisp files
> > I'm including in the dump,
> 
> I thought that was not supposed to be the case (the re-compiling should
> only be needed for *some* changes

Yes, you are right.

> tho I'm not exactly clear on what those are).

Additional primitives, stuff like that.  Not the kind of change which
was proposed.

Re: Objects that can't be purified during dumping

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 2275 bytes --]

On Sun, Jul 24, 2022, 11:07 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > changing the c source would require re-native-compiling all the elisp
> files
> > I'm including in the dump,
>
> I thought that was not supposed to be the case (the re-compiling should
> only be needed for *some* changes, tho I'm not exactly clear on what
> those are).


The hash is computed from the "major abi version" constant in the source,
the platform string, the list of subrs installed after the last
"symbols_of" procedure is called in main and (drum roll...) the options to
the configure script, with some modifications.  I haven't found where that
last step is done, so I'm not sure exactly how those are massaged, but at
least the -O and -g flags to gcc appear to be set to -O1 regardless of the
actual flags, so you can debug .elns produced by an optimized emacs.  It
would be nice if the build system facilitated making these multiple
versions, though.
On the other hand, if you do something like copy a configure command like
"../configure --without-native-compilation <other flags>" from the config
log of one build directory to configure in a second build directory, and
just add an option on the end:
""../configure --without-native-compilation <other flags>
--with-native-compilation"
Do your while build, etc, then decide to rebuild with -O0 -g3, but notice
how stupid having contradictory flags looks, so you rebuild using:
"../configure  <other flags>  --with-native-compilation",
Then the hash will be different and you won't be able to load the ELN files
compiled with the first build.
So, there could be some more normalization of the configure options, but
the current approach seems conservative.
I don't know if there is any implicit assumption that the compile is done
by an Emacs with only "loadup" in the dump.  I do know that when I ran the
build using a dump file containing all those additional core libraries,
something had made call-interactively advise itself (directly or
indirectly), so I was getting massive numbers of processes compiling
call_interactively_0 subrs.  So the subrs in the dump might be relevant to
the usability of the resulting ELN file, even though the compile is
otherwise done in a "clean" compile time environment.

Lynn


Lynn

[-- Attachment #2: Type: text/html, Size: 3138 bytes --]