Re: Preview: portable dumper

Re: Preview: portable dumper

[Angelo Graziosi]

Eli Zaretskii wrote:
>
> I'd urge people to try this branch and report any issues they see.

What should we look for?

  Angelo

Re: Preview: portable dumper

[Angelo Graziosi]

> Il 13 febbraio 2018 alle 23.06 Angelo Graziosi <angelo.g0@libero.it> ha scritto:
> 
> 
> Eli Zaretskii wrote:
> >
> > I'd urge people to try this branch and report any issues they see.
> 
> What should we look for?
> 

Meanwhile... here the build fails (MSYS2/MinGW64):

[...]
pdumper.c: In function 'dump_read_all':
pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
         read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
                                             ^~~~~~~~~~~~~
  CC       data.o
cc1.exe: some warnings being treated as errors
...

>   Angelo

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 107 bytes --]

Re: Preview: portable dumper

[Eli Zaretskii]

> Date: Tue, 13 Feb 2018 23:28:03 +0100 (CET)
> From: Angelo Graziosi <angelo.g0@libero.it>
> 
> pdumper.c: In function 'dump_read_all':
> pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
>          read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
>                                              ^~~~~~~~~~~~~

This is because the MS-Windows version of 'read' has 'unsigned int' as
its last argument.  But now I wonder why you don't see the same
problem in sysdep.c, where emacs_read does something very similar.
Not sure what am I missing here.

Re: Preview: portable dumper

[Angelo Graziosi]

Just for the record,

> Meanwhile... here the build fails (MSYS2/MinGW64):
> 
> [...]
> pdumper.c: In function 'dump_read_all':
> pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
>          read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
>                                              ^~~~~~~~~~~~~
>   CC       data.o
> cc1.exe: some warnings being treated as errors

I have seen you have commit many fixes but here the build still fails in the same manner.. Maybe you have not found the right solution yet.. 

Thanks,
  Angelo.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/16/2018 01:14 PM, Angelo Graziosi wrote:
> Just for the record,
> 
>> Meanwhile... here the build fails (MSYS2/MinGW64):
>>
>> [...]
>> pdumper.c: In function 'dump_read_all':
>> pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
>>           read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
>>                                               ^~~~~~~~~~~~~
>>    CC       data.o
>> cc1.exe: some warnings being treated as errors
> 
> I have seen you have commit many fixes but here the build still fails in the same manner.. Maybe you have not found the right solution yet..

On this one, I was waiting to see how the broader discussion went.

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Daniel Colascione <dancol@dancol.org>
> Date: Fri, 16 Feb 2018 13:25:00 -0800
> 
> >> pdumper.c: In function 'dump_read_all':
> >> pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
> >>           read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
> >>                                               ^~~~~~~~~~~~~
> >>    CC       data.o
> >> cc1.exe: some warnings being treated as errors
> > 
> > I have seen you have commit many fixes but here the build still fails in the same manner.. Maybe you have not found the right solution yet..
> 
> On this one, I was waiting to see how the broader discussion went.

OK, I've looked at the code, both in sysdep.c and in pdumper.c.  My
conclusion is that there's no real problem in sysdep.c, due to these
comments:

  /* Maximum number of bytes to read or write in a single system call.
     This works around a serious bug in Linux kernels before 2.6.16; see
     <https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=612839>.
     It's likely to work around similar bugs in other operating systems, so do it
     on all platforms.  Round INT_MAX down to a page size, with the conservative
     assumption that page sizes are at most 2**18 bytes (any kernel with a
     page size larger than that shouldn't have the bug).  */
  #ifndef MAX_RW_COUNT
  #define MAX_RW_COUNT (INT_MAX >> 18 << 18)
  #endif

  /* Read from FD to a buffer BUF with size NBYTE.
     If interrupted, process any quits and pending signals immediately
     if INTERRUPTIBLE, and then retry the read unless quitting.
     Return the number of bytes read, which might be less than NBYTE.
     On error, set errno to a value other than EINTR, and return -1.  */
  static ptrdiff_t
  emacs_intr_read (int fd, void *buf, ptrdiff_t nbyte, bool interruptible)
  {
    ssize_t result;

    /* There is no need to check against MAX_RW_COUNT, since no caller ever
       passes a size that large to emacs_read.  */
    do
      {
	if (interruptible)
	  maybe_quit ();
	result = read (fd, buf, nbyte);
      }
    while (result < 0 && errno == EINTR);

Since MAX_RW_COUNT is less than INT_MAX, we have no real problem here
for MS-Windows, as 'nbyte' will never overflow an unsigned int.  We
could add an eassert there, for Windows only, though, to make this
assumption explicit.

As for pdumper, we could do one of 2 things:

 . make a reasonable assumption that no .pdmp file will ever be larger
   than 2GB, change the assertion there which checks against SSIZE_MAX
   to check against UINT_MAX instead, and work internally with
   unsigned int instead of size_t counts; or

 . write a separate Windows-specific version of dump_read_all, which
   in the 64-bit build would limit to UINT_MAX the number of bytes we
   read on each iteration through the loop.

Daniel, which of these 2 alternatives do you prefer?  Or does anyone
have a better proposal?  (I already considered rewriting the code in
w32.c:sys_read so that it accepts a size_t last argument, and decided
it was unjustified for this single caller, as all the other direct
calls to 'read' in Emacs either use fixed small byte counts, or are
not compiled in the Windows build.)

Thanks.

Re: Preview: portable dumper

[Andy Moreton]

On Sat 17 Feb 2018, Eli Zaretskii wrote:
> Since MAX_RW_COUNT is less than INT_MAX, we have no real problem here
> for MS-Windows, as 'nbyte' will never overflow an unsigned int.  We
> could add an eassert there, for Windows only, though, to make this
> assumption explicit.

Comments mentioning MAX_RW_COUNT show that somebody checked an invariant
when writing the comment, but they do not guarantee that the invariant
will continue to be maintained in future versions of emacs. A check that
can be enforced by the compiler is preferable.

> As for pdumper, we could do one of 2 things:
>
>  . make a reasonable assumption that no .pdmp file will ever be larger
>    than 2GB, change the assertion there which checks against SSIZE_MAX
>    to check against UINT_MAX instead, and work internally with
>    unsigned int instead of size_t counts; or

This seems perfectly reasonable, and is the current implementation,
albeit checking for INT_MAX.

>  . write a separate Windows-specific version of dump_read_all, which
>    in the 64-bit build would limit to UINT_MAX the number of bytes we
>    read on each iteration through the loop.

This offers more opportunity for divergent implementations, so the other
option is preferable.

> Daniel, which of these 2 alternatives do you prefer?  Or does anyone
> have a better proposal?  (I already considered rewriting the code in
> w32.c:sys_read so that it accepts a size_t last argument, and decided
> it was unjustified for this single caller, as all the other direct
> calls to 'read' in Emacs either use fixed small byte counts, or are
> not compiled in the Windows build.)

Is there a gnulib replacement for read() that works for emacs, so that
we could use the proper prototype even on Windows ? If so we could get
rid of sys_read etc.

    AndyM

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Mon, 19 Feb 2018 22:23:17 +0000
> 
> Comments mentioning MAX_RW_COUNT show that somebody checked an invariant
> when writing the comment, but they do not guarantee that the invariant
> will continue to be maintained in future versions of emacs. A check that
> can be enforced by the compiler is preferable.

I added an assertion there (on master) that should make the code more
future-proof.

> > As for pdumper, we could do one of 2 things:
> >
> >  . make a reasonable assumption that no .pdmp file will ever be larger
> >    than 2GB, change the assertion there which checks against SSIZE_MAX
> >    to check against UINT_MAX instead, and work internally with
> >    unsigned int instead of size_t counts; or
> 
> This seems perfectly reasonable, and is the current implementation,
> albeit checking for INT_MAX.

No, the current code actually breaks the reads into chunks no larger
than INT_MAX.  So it doesn't assume a .pdmp file cannot be at most
2GB.

> >  . write a separate Windows-specific version of dump_read_all, which
> >    in the 64-bit build would limit to UINT_MAX the number of bytes we
> >    read on each iteration through the loop.
> 
> This offers more opportunity for divergent implementations, so the other
> option is preferable.

I presumed that we will not want to punish other platforms by making
them read piecemeal.  But if that's not an issue, I agree that the
second alternative is less elegant.

> Is there a gnulib replacement for read() that works for emacs, so that
> we could use the proper prototype even on Windows ?

No, because sys_read does much more than just fix the Windows _read.

Re: Preview: portable dumper

[Paul Eggert]

On 02/19/2018 02:23 PM, Andy Moreton wrote:
> Comments mentioning MAX_RW_COUNT show that somebody checked an invariant
> when writing the comment, but they do not guarantee that the invariant
> will continue to be maintained in future versions of emacs. A check that
> can be enforced by the compiler is preferable.

That "somebody" was me. That is, I looked at all the callers of these 
functions and checked that none of them go above MAX_RW_COUNT. That 
being said, it doesn't hurt to put in an eassert to check this as well 
(thanks, Eli). It would be a pain to check all the callers at 
compile-time, alas.

> Is there a gnulib replacement for read() that works for emacs, so that
> we could use the proper prototype even on Windows ?

There is. However, the Gnulib 'read' does some stuff that the Emacs 
'read' doesn't (and vice versa). Nobody has taken the time to try to 
unify them, and it's possible that unifying them would not be a good 
idea overall.

Re: Preview: portable dumper

[Eli Zaretskii]

> Date: Fri, 16 Feb 2018 22:14:18 +0100 (CET)
> From: Angelo Graziosi <angelo.g0@libero.it>
> 
> >          read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
> >                                              ^~~~~~~~~~~~~
> >   CC       data.o
> > cc1.exe: some warnings being treated as errors
> 
> I have seen you have commit many fixes but here the build still fails in the same manner.. Maybe you have not found the right solution yet.. 

We haven't.  Please be patient.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/13/2018 02:28 PM, Angelo Graziosi wrote:
> 
>> Il 13 febbraio 2018 alle 23.06 Angelo Graziosi <angelo.g0@libero.it> ha scritto:
>>
>>
>> Eli Zaretskii wrote:
>>>
>>> I'd urge people to try this branch and report any issues they see.
>>
>> What should we look for?
>>
> 
> Meanwhile... here the build fails (MSYS2/MinGW64):
> 
> [...]
> pdumper.c: In function 'dump_read_all':
> pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
>           read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
>                                               ^~~~~~~~~~~~~
>    CC       data.o
> cc1.exe: some warnings being treated as errors
> ...

Should be fixed.

Re: Preview: portable dumper

[Andy Moreton]

On Mon 19 Feb 2018, Daniel Colascione wrote:

> On 02/13/2018 02:28 PM, Angelo Graziosi wrote:
>>
>>> Il 13 febbraio 2018 alle 23.06 Angelo Graziosi <angelo.g0@libero.it> ha scritto:
>>>
>>>
>
>>> Eli Zaretskii wrote:
>>>>
>>>> I'd urge people to try this branch and report any issues they see.
>>>
>>> What should we look for?
>>>
>>
>> Meanwhile... here the build fails (MSYS2/MinGW64):
>>
>> [...]
>> pdumper.c: In function 'dump_read_all':
>> pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
>>           read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
>>                                               ^~~~~~~~~~~~~
>>    CC       data.o
>> cc1.exe: some warnings being treated as errors
>> ...
>
> Should be fixed.

The patch for this in commit b7da1a5af8f1 is not quite right:

+      /* Some platforms accept only int-sized values to read.  */
+      unsigned chunk_to_read = UINT_MAX;
+      if (bytes_to_read - bytes_read < chunk_to_read)
+        chunk_to_read = (unsigned)(bytes_to_read - bytes_read);

Windows platforms use sys_read from w32.c:
    int sys_read(int fd, char *buffer, unsigned int count);

The limit for a chunk should be INT_MAX (not UINT_MAX) so that it
is representable in the (signed) return value.

    AndyM

Re: Preview: portable dumper

[Daniel Colascione]

On 02/19/2018 12:03 PM, Andy Moreton wrote:
> On Mon 19 Feb 2018, Daniel Colascione wrote:
> 
>> On 02/13/2018 02:28 PM, Angelo Graziosi wrote:
>>>
>>>> Il 13 febbraio 2018 alle 23.06 Angelo Graziosi <angelo.g0@libero.it> ha scritto:
>>>>
>>>>
>>
>>>> Eli Zaretskii wrote:
>>>>>
>>>>> I'd urge people to try this branch and report any issues they see.
>>>>
>>>> What should we look for?
>>>>
>>>
>>> Meanwhile... here the build fails (MSYS2/MinGW64):
>>>
>>> [...]
>>> pdumper.c: In function 'dump_read_all':
>>> pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
>>>            read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
>>>                                                ^~~~~~~~~~~~~
>>>     CC       data.o
>>> cc1.exe: some warnings being treated as errors
>>> ...
>>
>> Should be fixed.
> 
> The patch for this in commit b7da1a5af8f1 is not quite right:
> 
> +      /* Some platforms accept only int-sized values to read.  */
> +      unsigned chunk_to_read = UINT_MAX;
> +      if (bytes_to_read - bytes_read < chunk_to_read)
> +        chunk_to_read = (unsigned)(bytes_to_read - bytes_read);
> 
> Windows platforms use sys_read from w32.c:
>      int sys_read(int fd, char *buffer, unsigned int count);
> 
> The limit for a chunk should be INT_MAX (not UINT_MAX) so that it
> is representable in the (signed) return value.

Is the return value int too? Or is it a ssize_t that can hold UINT_MAX? 
If the former, agreed.

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Mon, 19 Feb 2018 20:03:09 +0000
> 
>     int sys_read(int fd, char *buffer, unsigned int count);
> 
> The limit for a chunk should be INT_MAX (not UINT_MAX) so that it
> is representable in the (signed) return value.

We should be able to change sys_read to return ssize_t instead.  Can
you try that?

Re: Preview: portable dumper

[Daniel Colascione]

On 02/19/2018 12:18 PM, Eli Zaretskii wrote:
>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> Date: Mon, 19 Feb 2018 20:03:09 +0000
>>
>>      int sys_read(int fd, char *buffer, unsigned int count);
>>
>> The limit for a chunk should be INT_MAX (not UINT_MAX) so that it
>> is representable in the (signed) return value.
> 
> We should be able to change sys_read to return ssize_t instead.  Can
> you try that?

I looked at the w32 code --- we give the size to _read directly in the 
non-pipe, non-socket, non-serial case, and MSDN doesn't say what happens 
when size > INT_MAX, so in the interest of being rather safe than sorry, 
I just changed the pdumper code to use INT_MAX. It shouldn't matter in 
practice.

Re: Preview: portable dumper

[Angelo Graziosi]

> Il 19 febbraio 2018 alle 18.04 Daniel Colascione ha scritto:
> 
> 
> On 02/13/2018 02:28 PM, Angelo Graziosi wrote:
> > 
> > [...]
> > pdumper.c: In function 'dump_read_all':
> > pdumper.c:4723:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
> >           read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
> >                                               ^~~~~~~~~~~~~
> >    CC       data.o
> > cc1.exe: some warnings being treated as errors
> > ...
> 
> Should be fixed.

Now it builds! 

Thanks,
  Angelo.

Re: Preview: portable dumper

[Angelo Graziosi]

> Il 19 febbraio 2018 alle 18.04 Daniel Colascione ha scritto:
> 
> Should be fixed.

...and now commit aacc1390f21ddb003c9d6a786597e89284bafec3 is broken..

[...]
  CC       keyboard.o
emacs.c: In function 'main':
emacs.c:1849:3: error: 'else' without a previous 'if'
   else
   ^~~~
make[1]: *** [Makefile:387: emacs.o] Error 1
make[1]: *** Attesa per i processi non terminati....
make[1]: uscita dalla directory "/tmp/emacs-pdumper/src"
make: *** [Makefile:420: src] Error 2

Re: Preview: portable dumper

[Daniel Colascione]

On 02/26/2018 04:03 AM, Angelo Graziosi wrote:
> 
>> Il 19 febbraio 2018 alle 18.04 Daniel Colascione ha scritto:
>>
>> Should be fixed.
> 
> ...and now commit aacc1390f21ddb003c9d6a786597e89284bafec3 is broken..
> 
> [...]
>    CC       keyboard.o
> emacs.c: In function 'main':
> emacs.c:1849:3: error: 'else' without a previous 'if'
>     else
>     ^~~~
> make[1]: *** [Makefile:387: emacs.o] Error 1
> make[1]: *** Attesa per i processi non terminati....
> make[1]: uscita dalla directory "/tmp/emacs-pdumper/src"
> make: *** [Makefile:420: src] Error 2

Now that's fixed too. I wish we had better CI.

Re: Preview: portable dumper

[Angelo Graziosi]

They are 4 weeks that pdumper branch is not synchronized, should we continue to test it? 

During this time, builds of that branch have been tested on Windows, GNU/Linux and macOS without showing any specific issue...

Angelo

Re: Preview: portable dumper

[Daniel Colascione]

On 03/29/2018 12:12 AM, Angelo Graziosi wrote:
> 
> They are 4 weeks that pdumper branch is not synchronized, should we continue to test it?
> 
> During this time, builds of that branch have been tested on Windows, GNU/Linux and macOS without showing any specific issue...

I'm not sure what's left to test. It works. What's stopping a merge?

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

> On 03/29/2018 12:12 AM, Angelo Graziosi wrote:
>>
>> They are 4 weeks that pdumper branch is not synchronized, should we continue to test it?
>>
>> During this time, builds of that branch have been tested on Windows, GNU/Linux and macOS without showing any specific issue...
>
> I'm not sure what's left to test. It works. What's stopping a merge?

I just test-merged it to master, with only two issues:

1. a merge conflict in src/nsfont.m as a result of 7ff62ed221c
2. The Lisp_Buffer_Local_Value hash has changed, although itʼs not
clear to me why, since I only see comment changes in its definition. I
updated dmpstruct.h, and the build works fine (it runs gnus, as proven
by this message).

Robert

Re: Preview: portable dumper

[Pip Cet]

I've just skimmed the diff, and I think the change is a significant
improvement, and will help with my GC experiments even though I'm
probably never going to be able to use the portable dumper. Some minor
comments:

it is not clear to me why some staticpro's are moved to happen after
the initialisation of the variable and some aren't. Maybe we should
change staticpro to have a
void staticpro(Lisp_Object *ptr, Lisp_Object initial_value);
signature and save some lines of code?

I don't understand this hunk in sysdep.c:

@@ -2138,7 +2149,7 @@ init_signals (bool dumping)
 #ifdef SIGUSR2
   add_user_signal (SIGUSR2, "sigusr2");
 #endif
-  sigaction (SIGABRT, &thread_fatal_action, 0);
+  //sigaction (SIGABRT, &thread_fatal_action, 0);
 #ifdef SIGPRE
   sigaction (SIGPRE, &thread_fatal_action, 0);
 #endif

Similarly, this hunk in pcase.el:

@@ -63,6 +63,7 @@
 ;; FIXME: Now that macroexpansion is also performed when loading an interpreted
 ;; file, this is not a real problem any more.
 (defconst pcase--memoize (make-hash-table :weakness 'key :test 'eq))
+;; (defconst pcase--memoize (make-hash-table :test 'eq))
 ;; (defconst pcase--memoize-1 (make-hash-table :test 'eq))
 ;; (defconst pcase--memoize-2 (make-hash-table :weakness 'key :test 'equal))

Are those two intentional?

I would vaguely prefer if "pdumper" names were limited to the actual
pdumper implementation, and "dumper" names were used instead in code
that does not depend on the implementation of the dumper. So we'd have
dumper_object_p(), defined to false if there's no dumper, and
pdumper_object_p otherwise. This would make things slightly easier for
(a) competing implementations (b) abusing the DUMPER_* machinery for
GC experiments (c) possibly extending pdumper to combine several
images rather than use a single one. It would also help avoid _impl
names.

I think it's worth mentioning that staticpro is now required for
variables that used to be safe: this change comes at the price of a
tiny performance hit because of the additional GC roots, but it's
worth it, IMHO.

Apart from that, there appear to be no disadvantages for people who
prefer to continue using unexec. Is that correct?

On Thu, Mar 29, 2018 at 9:39 AM, Robert Pluim <rpluim@gmail.com> wrote:
> Daniel Colascione <dancol@dancol.org> writes:
>
>> On 03/29/2018 12:12 AM, Angelo Graziosi wrote:
>>>
>>> They are 4 weeks that pdumper branch is not synchronized, should we continue to test it?
>>>
>>> During this time, builds of that branch have been tested on Windows, GNU/Linux and macOS without showing any specific issue...
>>
>> I'm not sure what's left to test. It works. What's stopping a merge?
>
> I just test-merged it to master, with only two issues:
>
> 1. a merge conflict in src/nsfont.m as a result of 7ff62ed221c
> 2. The Lisp_Buffer_Local_Value hash has changed, although itʼs not
> clear to me why, since I only see comment changes in its definition. I
> updated dmpstruct.h, and the build works fine (it runs gnus, as proven
> by this message).
>
> Robert
>

Re: Preview: portable dumper

[Daniel Colascione]

On 03/29/2018 06:35 AM, Pip Cet wrote:
> I've just skimmed the diff, and I think the change is a significant
> improvement, and will help with my GC experiments even though I'm
> probably never going to be able to use the portable dumper.

Why not? I eventually want to remove unexec entirely.

> Some minor
> comments:
> 
> it is not clear to me why some staticpro's are moved to happen after
> the initialisation of the variable and some aren't.

The code has always been a bit unclear about that; we don't GC that 
early, so it doesn't really matter- -- but if we did, we'd be saved by 
nil being all zero these days.

> Maybe we should
> change staticpro to have a
> void staticpro(Lisp_Object *ptr, Lisp_Object initial_value);
> signature and save some lines of code?

Sometimes the value from .data is perfectly good though, and at that 
point, the initial_value setting would be a waste.

> I don't understand this hunk in sysdep.c:
> 
> @@ -2138,7 +2149,7 @@ init_signals (bool dumping)
>   #ifdef SIGUSR2
>     add_user_signal (SIGUSR2, "sigusr2");
>   #endif
> -  sigaction (SIGABRT, &thread_fatal_action, 0);
> +  //sigaction (SIGABRT, &thread_fatal_action, 0);
>   #ifdef SIGPRE
>     sigaction (SIGPRE, &thread_fatal_action, 0);
>   #endif

Thanks; I'll clean that up. That's left over from the demand paging 
experiment.

> 
> Similarly, this hunk in pcase.el:
> 
> @@ -63,6 +63,7 @@
>   ;; FIXME: Now that macroexpansion is also performed when loading an interpreted
>   ;; file, this is not a real problem any more.
>   (defconst pcase--memoize (make-hash-table :weakness 'key :test 'eq))
> +;; (defconst pcase--memoize (make-hash-table :test 'eq))
>   ;; (defconst pcase--memoize-1 (make-hash-table :test 'eq))
>   ;; (defconst pcase--memoize-2 (make-hash-table :weakness 'key :test 'equal))
> 

Thanks for catching that.

> Are those two intentional?
> 
> I would vaguely prefer if "pdumper" names were limited to the actual
> pdumper implementation, and "dumper" names were used instead in code
> that does not depend on the implementation of the dumper. So we'd have
> dumper_object_p(), defined to false if there's no dumper, and
> pdumper_object_p otherwise.

I'd rather not do a big renaming at this point. Nothing stops someone 
doing cleanup work post-merge if sufficiently motivated.

> Apart from that, there appear to be no disadvantages for people who
> prefer to continue using unexec. Is that correct?

That's correct.

Re: Preview: portable dumper

[Pip Cet]

On Thu, Mar 29, 2018 at 3:31 PM, Daniel Colascione <dancol@dancol.org> wrote:
> On 03/29/2018 06:35 AM, Pip Cet wrote:
>> I've just skimmed the diff, and I think the change is a significant
>> improvement, and will help with my GC experiments even though I'm
>> probably never going to be able to use the portable dumper.
>
>
> Why not? I eventually want to remove unexec entirely.

No objections to removing unexec, as long as temacs remains an option
:-) I haven't thought about dumping/restoring SpiderMonkey state, but
as it's hairy enough to get things working with setjmp(), I'd rather
not worry about it now.

I do have a half-finished (working, but not thoroughly tested) unexec
for asm.js, too, which sped things up somewhat, but I don't see why
the portable dumper wouldn't work there, too.

>
>> Some minor
>> comments:
>>
>> it is not clear to me why some staticpro's are moved to happen after
>> the initialisation of the variable and some aren't.
>
>
> The code has always been a bit unclear about that; we don't GC that early,
> so it doesn't really matter- -- but if we did, we'd be saved by nil being
> all zero these days.

Well, Qnil isn't all zero (or even constant) here, and I do GC
unpredictably, though hopefully never that early; I'm still safe if
the staticpro goes first: the all-zero value, representing 0.0, is
fine for GC.

>> Maybe we should
>> change staticpro to have a
>> void staticpro(Lisp_Object *ptr, Lisp_Object initial_value);
>> signature and save some lines of code?
>
>
> Sometimes the value from .data is perfectly good though, and at that point,
> the initial_value setting would be a waste.

... as are explicit Qnil initializations, in general. So

inline void staticpro(Lisp_Object *ptr, Lisp_Object initial_value)
{
  assume(*ptr == Qnil);
  real_staticpro(ptr);
  if (initial_value != Qnil)
    *ptr = initial_value;
}

would actually reduce code size. But, again, it'd be better to merge
first, then optimize such minor things.

>> I would vaguely prefer if "pdumper" names were limited to the actual
>> pdumper implementation, and "dumper" names were used instead in code
>> that does not depend on the implementation of the dumper. So we'd have
>> dumper_object_p(), defined to false if there's no dumper, and
>> pdumper_object_p otherwise.
>
>
> I'd rather not do a big renaming at this point. Nothing stops someone doing
> cleanup work post-merge if sufficiently motivated.

Agreed.

>> Apart from that, there appear to be no disadvantages for people who
>> prefer to continue using unexec. Is that correct?
> That's correct.

Thanks!

Re: Preview: portable dumper

[Pip Cet]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

On Thu, Mar 29, 2018 at 4:15 PM, Pip Cet <pipcet@gmail.com> wrote:
> inline void staticpro(Lisp_Object *ptr, Lisp_Object initial_value)
> {
>   assume(*ptr == Qnil);
>   real_staticpro(ptr);
>   if (initial_value != Qnil)
>     *ptr = initial_value;
> }

Okay, GCC doesn't appear to optimize that expression by default, but
I've attached a patch which does keep stripped binary size constant.

It reduces C code size a little, avoids the ambiguity of whether
staticpro or the initialization goes first, removes the Qnil == 0
assumption from a few places and generally seems like a slight
improvement to me. Totally independent from the pdumper patch, of
course. (And I'd prefer the pdumper patch to go in first.)

[-- Attachment #2: emacs-staticpro-001.diff --]
[-- Type: text/x-patch, Size: 46971 bytes --]

diff --git a/src/alloc.c b/src/alloc.c
index 369592d70ee..ff32d1639ee 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -5696,7 +5696,7 @@ purecopy (Lisp_Object obj)
    VARADDRESS.  */
 
 void
-staticpro (Lisp_Object *varaddress)
+staticpro_1 (Lisp_Object *varaddress)
 {
   if (staticidx >= NSTATICS)
     fatal ("NSTATICS too small; try increasing and recompiling Emacs.");
diff --git a/src/bidi.c b/src/bidi.c
index 9bc8dbe8603..24e435bbc1d 100644
--- a/src/bidi.c
+++ b/src/bidi.c
@@ -1090,25 +1090,20 @@ bidi_unshelve_cache (void *databuf, bool just_free)
 static void
 bidi_initialize (void)
 {
-  bidi_type_table = uniprop_table (intern ("bidi-class"));
+  staticpro (&bidi_type_table, uniprop_table (intern ("bidi-class")));
   if (NILP (bidi_type_table))
     emacs_abort ();
-  staticpro (&bidi_type_table);
 
-  bidi_mirror_table = uniprop_table (intern ("mirroring"));
+  staticpro (&bidi_mirror_table, uniprop_table (intern ("mirroring")));
   if (NILP (bidi_mirror_table))
     emacs_abort ();
-  staticpro (&bidi_mirror_table);
 
-  bidi_brackets_table = uniprop_table (intern ("bracket-type"));
+  staticpro (&bidi_brackets_table, uniprop_table (intern ("bracket-type")));
   if (NILP (bidi_brackets_table))
     emacs_abort ();
-  staticpro (&bidi_brackets_table);
 
-  paragraph_start_re = build_string ("^\\(\f\\|[ \t]*\\)$");
-  staticpro (&paragraph_start_re);
-  paragraph_separate_re = build_string ("^[ \t\f]*$");
-  staticpro (&paragraph_separate_re);
+  staticpro (&paragraph_start_re, build_string ("^\\(\f\\|[ \t]*\\)$"));
+  staticpro (&paragraph_separate_re, build_string ("^[ \t\f]*$"));
 
   bidi_cache_sp = 0;
   bidi_cache_total_alloc = 0;
diff --git a/src/buffer.c b/src/buffer.c
index 14837372d34..e90bf6ece20 100644
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -5441,12 +5441,11 @@ defvar_per_buffer (struct Lisp_Buffer_Objfwd *bo_fwd, const char *namestring,
 void
 syms_of_buffer (void)
 {
-  staticpro (&last_overlay_modification_hooks);
-  last_overlay_modification_hooks
-    = Fmake_vector (make_number (10), Qnil);
+  staticpro (&last_overlay_modification_hooks,
+             Fmake_vector (make_number (10), Qnil));
 
-  staticpro (&QSFundamental);
-  staticpro (&Vbuffer_alist);
+  staticpro (&QSFundamental, Qnil);
+  staticpro (&Vbuffer_alist, Qnil);
 
   DEFSYM (Qchoice, "choice");
   DEFSYM (Qleft, "left");
diff --git a/src/callint.c b/src/callint.c
index 08a8bba4646..8cac305e0b4 100644
--- a/src/callint.c
+++ b/src/callint.c
@@ -826,11 +826,9 @@ Its numeric meaning is what you would get from `(interactive "p")'.  */)
 void
 syms_of_callint (void)
 {
-  point_marker = Fmake_marker ();
-  staticpro (&point_marker);
+  staticpro (&point_marker, Fmake_marker ());
 
-  callint_message = Qnil;
-  staticpro (&callint_message);
+  staticpro (&callint_message, Qnil);
 
   preserved_fns = listn (CONSTYPE_PURE, 4,
 			 intern_c_string ("region-beginning"),
diff --git a/src/callproc.c b/src/callproc.c
index 973f324139c..5b3867288e1 100644
--- a/src/callproc.c
+++ b/src/callproc.c
@@ -1640,15 +1640,13 @@ void
 syms_of_callproc (void)
 {
 #ifndef DOS_NT
-  Vtemp_file_name_pattern = build_string ("emacsXXXXXX");
+  staticpro (&Vtemp_file_name_pattern, build_string ("emacsXXXXXX"));
 #else  /* DOS_NT */
-  Vtemp_file_name_pattern = build_string ("emXXXXXX");
+  staticpro (&Vtemp_file_name_pattern, build_string ("emXXXXXX"));
 #endif
-  staticpro (&Vtemp_file_name_pattern);
 
 #ifdef MSDOS
-  synch_process_tempfile = make_number (0);
-  staticpro (&synch_process_tempfile);
+  staticpro (&synch_process_tempfile, make_number (0));
 #endif
 
   DEFVAR_LISP ("shell-file-name", Vshell_file_name,
diff --git a/src/casetab.c b/src/casetab.c
index 8f806a0647c..d2388d5fc58 100644
--- a/src/casetab.c
+++ b/src/casetab.c
@@ -247,6 +247,11 @@ init_casetab_once (void)
   DEFSYM (Qcase_table, "case-table");
   Fput (Qcase_table, Qchar_table_extra_slots, make_number (3));
 
+  staticpro (&Vascii_canon_table, Qnil);
+  staticpro (&Vascii_downcase_table, Qnil);
+  staticpro (&Vascii_eqv_table, Qnil);
+  staticpro (&Vascii_upcase_table, Qnil);
+
   down = Fmake_char_table (Qcase_table, Qnil);
   Vascii_downcase_table = down;
   set_char_table_purpose (down, Qcase_table);
@@ -289,11 +294,6 @@ syms_of_casetab (void)
 {
   DEFSYM (Qcase_table_p, "case-table-p");
 
-  staticpro (&Vascii_canon_table);
-  staticpro (&Vascii_downcase_table);
-  staticpro (&Vascii_eqv_table);
-  staticpro (&Vascii_upcase_table);
-
   defsubr (&Scase_table_p);
   defsubr (&Scurrent_case_table);
   defsubr (&Sstandard_case_table);
diff --git a/src/ccl.c b/src/ccl.c
index ed8588d7f8a..b2a45c87419 100644
--- a/src/ccl.c
+++ b/src/ccl.c
@@ -2274,8 +2274,7 @@ Return index number of the registered map.  */)
 void
 syms_of_ccl (void)
 {
-  staticpro (&Vccl_program_table);
-  Vccl_program_table = Fmake_vector (make_number (32), Qnil);
+  staticpro (&Vccl_program_table, Fmake_vector (make_number (32), Qnil));
 
   DEFSYM (Qccl, "ccl");
   DEFSYM (Qcclp, "cclp");
diff --git a/src/character.c b/src/character.c
index 6a689808043..cbde3f414e9 100644
--- a/src/character.c
+++ b/src/character.c
@@ -1099,8 +1099,7 @@ syms_of_character (void)
   DEFSYM (Qcharacterp, "characterp");
   DEFSYM (Qauto_fill_chars, "auto-fill-chars");
 
-  staticpro (&Vchar_unify_table);
-  Vchar_unify_table = Qnil;
+  staticpro (&Vchar_unify_table, Qnil);
 
   defsubr (&Smax_char);
   defsubr (&Scharacterp);
diff --git a/src/charset.c b/src/charset.c
index 05290e86b4e..60b46137d19 100644
--- a/src/charset.c
+++ b/src/charset.c
@@ -2348,17 +2348,13 @@ syms_of_charset (void)
   DEFSYM (Qeight_bit, "eight-bit");
   DEFSYM (Qiso_8859_1, "iso-8859-1");
 
-  staticpro (&Vcharset_ordered_list);
-  Vcharset_ordered_list = Qnil;
+  staticpro (&Vcharset_ordered_list, Qnil);
 
-  staticpro (&Viso_2022_charset_list);
-  Viso_2022_charset_list = Qnil;
+  staticpro (&Viso_2022_charset_list, Qnil);
 
-  staticpro (&Vemacs_mule_charset_list);
-  Vemacs_mule_charset_list = Qnil;
+  staticpro (&Vemacs_mule_charset_list, Qnil);
 
-  staticpro (&Vcharset_hash_table);
-  Vcharset_hash_table = CALLN (Fmake_hash_table, QCtest, Qeq);
+  staticpro (&Vcharset_hash_table, CALLN (Fmake_hash_table, QCtest, Qeq));
 
   charset_table = charset_table_init;
   charset_table_size = ARRAYELTS (charset_table_init);
diff --git a/src/coding.c b/src/coding.c
index a16142a9b41..44c68e8b880 100644
--- a/src/coding.c
+++ b/src/coding.c
@@ -10818,20 +10818,15 @@ init_coding_once (void)
 void
 syms_of_coding (void)
 {
-  staticpro (&Vcoding_system_hash_table);
-  Vcoding_system_hash_table = CALLN (Fmake_hash_table, QCtest, Qeq);
+  staticpro (&Vcoding_system_hash_table, CALLN (Fmake_hash_table, QCtest, Qeq));
 
-  staticpro (&Vsjis_coding_system);
-  Vsjis_coding_system = Qnil;
+  staticpro (&Vsjis_coding_system, Qnil);
 
-  staticpro (&Vbig5_coding_system);
-  Vbig5_coding_system = Qnil;
+  staticpro (&Vbig5_coding_system, Qnil);
 
-  staticpro (&Vcode_conversion_reused_workbuf);
-  Vcode_conversion_reused_workbuf = Qnil;
+  staticpro (&Vcode_conversion_reused_workbuf, Qnil);
 
-  staticpro (&Vcode_conversion_workbuf_name);
-  Vcode_conversion_workbuf_name = build_pure_c_string (" *code-conversion-work*");
+  staticpro (&Vcode_conversion_workbuf_name, build_pure_c_string (" *code-conversion-work*"));
 
   reused_workbuf_in_use = 0;
 
@@ -10914,9 +10909,7 @@ syms_of_coding (void)
   DEFSYM (QCpre_write_conversion, ":pre-write-conversion");
   DEFSYM (QCascii_compatible_p, ":ascii-compatible-p");
 
-  Vcoding_category_table
-    = Fmake_vector (make_number (coding_category_max), Qnil);
-  staticpro (&Vcoding_category_table);
+  staticpro (&Vcoding_category_table, Fmake_vector (make_number (coding_category_max), Qnil));
   /* Followings are target of code detection.  */
   ASET (Vcoding_category_table, coding_category_iso_7,
 	intern_c_string ("coding-category-iso-7"));
@@ -11352,10 +11345,9 @@ internal character representation.  */);
     Fset (AREF (Vcoding_category_table, i), Qno_conversion);
 
 #if defined (DOS_NT)
-  system_eol_type = Qdos;
+  staticpro (&system_eol_type, Qdos);
 #else
-  system_eol_type = Qunix;
+  staticpro (&system_eol_type, Qunix);
 #endif
-  staticpro (&system_eol_type);
 }
 #endif /* emacs */
diff --git a/src/composite.c b/src/composite.c
index 746c2959f84..f7e95a0255b 100644
--- a/src/composite.c
+++ b/src/composite.c
@@ -1907,19 +1907,15 @@ syms_of_composite (void)
      and thus it's not worth to save memory in such a way.  So, we
      make the table not weak.  */
   Lisp_Object args[] = {QCtest, Qequal, QCsize, make_number (311)};
-  composition_hash_table = CALLMANY (Fmake_hash_table, args);
-  staticpro (&composition_hash_table);
+  staticpro (&composition_hash_table, CALLMANY (Fmake_hash_table, args));
 
   /* Make a hash table for glyph-string.  */
-  gstring_hash_table = CALLMANY (Fmake_hash_table, args);
-  staticpro (&gstring_hash_table);
+  staticpro (&gstring_hash_table, CALLMANY (Fmake_hash_table, args));
 
-  staticpro (&gstring_work_headers);
-  gstring_work_headers = make_uninit_vector (8);
+  staticpro (&gstring_work_headers, make_uninit_vector (8));
   for (i = 0; i < 8; i++)
     ASET (gstring_work_headers, i, Fmake_vector (make_number (i + 2), Qnil));
-  staticpro (&gstring_work);
-  gstring_work = Fmake_vector (make_number (10), Qnil);
+  staticpro (&gstring_work, Fmake_vector (make_number (10), Qnil));
 
   /* Text property `composition' should be nonsticky by default.  */
   Vtext_property_default_nonsticky
diff --git a/src/dbusbind.c b/src/dbusbind.c
index ec3707d18f3..3af20551993 100644
--- a/src/dbusbind.c
+++ b/src/dbusbind.c
@@ -1824,8 +1824,7 @@ be called when the D-Bus reply message arrives.  */);
 #endif
 
   /* Initialize internal objects.  */
-  xd_registered_buses = Qnil;
-  staticpro (&xd_registered_buses);
+  staticpro (&xd_registered_buses, Qnil);
 
   Fprovide (intern_c_string ("dbusbind"), Qnil);
 
diff --git a/src/dispnew.c b/src/dispnew.c
index 56f125218dc..608e1123459 100644
--- a/src/dispnew.c
+++ b/src/dispnew.c
@@ -6227,8 +6227,7 @@ syms_of_display (void)
   defsubr (&Sdump_redisplay_history);
 #endif
 
-  frame_and_buffer_state = Fmake_vector (make_number (20), Qlambda);
-  staticpro (&frame_and_buffer_state);
+  staticpro (&frame_and_buffer_state, Fmake_vector (make_number (20), Qlambda));
 
   /* This is the "purpose" slot of a display table.  */
   DEFSYM (Qdisplay_table, "display-table");
diff --git a/src/emacs.c b/src/emacs.c
index 8ea61b71fb7..3229c4d4648 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -2680,7 +2680,6 @@ libraries; only those already known by Emacs will be loaded.  */);
   Fput (intern_c_string ("dynamic-library-alist"), Qrisky_local_variable, Qt);
 
 #ifdef WINDOWSNT
-  Vlibrary_cache = Qnil;
-  staticpro (&Vlibrary_cache);
+  staticpro (&Vlibrary_cache, Qnil);
 #endif
 }
diff --git a/src/eval.c b/src/eval.c
index a6e1d86c4ab..ba304ee8456 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -4107,13 +4107,10 @@ alist of active lexical bindings.  */);
      (Just imagine if someone makes it buffer-local).  */
   Funintern (Qinternal_interpreter_environment, Qnil);
 
-  Vrun_hooks = intern_c_string ("run-hooks");
-  staticpro (&Vrun_hooks);
+  staticpro (&Vrun_hooks, intern_c_string ("run-hooks"));
 
-  staticpro (&Vautoload_queue);
-  Vautoload_queue = Qnil;
-  staticpro (&Vsignaling_function);
-  Vsignaling_function = Qnil;
+  staticpro (&Vautoload_queue, Qnil);
+  staticpro (&Vsignaling_function, Qnil);
 
   inhibit_lisp_code = Qnil;
 
diff --git a/src/fileio.c b/src/fileio.c
index 52ca8b6297e..17230a82fb0 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -6091,8 +6091,7 @@ annotation functions in `write-region-annotate-functions' changed the
 current buffer, the function stored in this variable is called for
 each of those additional buffers as well, in addition to the original
 buffer.  The relevant buffer is current during each function call.  */);
-  Vwrite_region_post_annotation_function = Qnil;
-  staticpro (&Vwrite_region_annotation_buffers);
+  staticpro (&Vwrite_region_annotation_buffers, Qnil);
 
   DEFVAR_LISP ("write-region-annotations-so-far",
 	       Vwrite_region_annotations_so_far,
diff --git a/src/fns.c b/src/fns.c
index 94b9d984f0d..16c842b7a12 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -5178,11 +5178,9 @@ compilation.  */);
   Voverriding_plist_environment = Qnil;
   DEFSYM (Qoverriding_plist_environment, "overriding-plist-environment");
 
-  staticpro (&string_char_byte_cache_string);
-  string_char_byte_cache_string = Qnil;
+  staticpro (&string_char_byte_cache_string, Qnil);
 
-  require_nesting_list = Qnil;
-  staticpro (&require_nesting_list);
+  staticpro (&require_nesting_list, Qnil);
 
   Fset (Qyes_or_no_p_history, Qnil);
 
diff --git a/src/font.c b/src/font.c
index a6d3f5d4798..34ff23b40d0 100644
--- a/src/font.c
+++ b/src/font.c
@@ -5307,8 +5307,7 @@ syms_of_font (void)
   sort_shift_bits[FONT_WIDTH_INDEX] = 23;
   /* Note that the other elements in sort_shift_bits are not used.  */
 
-  staticpro (&font_charset_alist);
-  font_charset_alist = Qnil;
+  staticpro (&font_charset_alist, Qnil);
 
   DEFSYM (Qopentype, "opentype");
 
@@ -5346,13 +5345,10 @@ syms_of_font (void)
 
   DEFSYM (QCuser_spec, ":user-spec");
 
-  staticpro (&scratch_font_spec);
-  scratch_font_spec = Ffont_spec (0, NULL);
-  staticpro (&scratch_font_prefer);
-  scratch_font_prefer = Ffont_spec (0, NULL);
+  staticpro (&scratch_font_spec, Ffont_spec (0, NULL));
+  staticpro (&scratch_font_prefer, Ffont_spec (0, NULL));
 
-  staticpro (&Vfont_log_deferred);
-  Vfont_log_deferred = Fmake_vector (make_number (3), Qnil);
+  staticpro (&Vfont_log_deferred, Fmake_vector (make_number (3), Qnil));
 
 #if 0
 #ifdef HAVE_LIBOTF
@@ -5447,8 +5443,7 @@ This variable cannot be set; trying to do so will signal an error.  */);
   Vfont_width_table = BUILD_STYLE_TABLE (width_table);
   make_symbol_constant (intern_c_string ("font-width-table"));
 
-  staticpro (&font_style_table);
-  font_style_table = make_uninit_vector (3);
+  staticpro (&font_style_table, make_uninit_vector (3));
   ASET (font_style_table, 0, Vfont_weight_table);
   ASET (font_style_table, 1, Vfont_slant_table);
   ASET (font_style_table, 2, Vfont_width_table);
diff --git a/src/fontset.c b/src/fontset.c
index 6ca64068717..71dd45c3237 100644
--- a/src/fontset.c
+++ b/src/fontset.c
@@ -2116,14 +2116,11 @@ syms_of_fontset (void)
   DEFSYM (Qappend, "append");
   DEFSYM (Qlatin, "latin");
 
-  Vcached_fontset_data = Qnil;
-  staticpro (&Vcached_fontset_data);
+  staticpro (&Vcached_fontset_data, Qnil);
 
-  Vfontset_table = Fmake_vector (make_number (32), Qnil);
-  staticpro (&Vfontset_table);
+  staticpro (&Vfontset_table, Fmake_vector (make_number (32), Qnil));
 
-  Vdefault_fontset = Fmake_char_table (Qfontset, Qnil);
-  staticpro (&Vdefault_fontset);
+  staticpro (&Vdefault_fontset, Fmake_char_table (Qfontset, Qnil));
   set_fontset_id (Vdefault_fontset, make_number (0));
   set_fontset_name
     (Vdefault_fontset,
@@ -2131,8 +2128,7 @@ syms_of_fontset (void)
   ASET (Vfontset_table, 0, Vdefault_fontset);
   next_fontset_id = 1;
 
-  auto_fontset_alist = Qnil;
-  staticpro (&auto_fontset_alist);
+  staticpro (&auto_fontset_alist, Qnil);
 
   DEFVAR_LISP ("font-encoding-charset-alist", Vfont_encoding_charset_alist,
 	       doc: /*
diff --git a/src/frame.c b/src/frame.c
index 86caa32615d..40abef3a66b 100644
--- a/src/frame.c
+++ b/src/frame.c
@@ -6092,7 +6092,7 @@ making the child frame unresponsive to user actions, the default is to
 iconify the top level frame instead.  */);
   iconify_child_frame = Qiconify_top_level;
 
-  staticpro (&Vframe_list);
+  staticpro (&Vframe_list, Qnil);
 
   defsubr (&Sframep);
   defsubr (&Sframe_live_p);
diff --git a/src/ftfont.c b/src/ftfont.c
index c2e093e633d..eaef4574b4e 100644
--- a/src/ftfont.c
+++ b/src/ftfont.c
@@ -2735,16 +2735,13 @@ syms_of_ftfont (void)
   DEFSYM (Qsans, "sans");
   DEFSYM (Qsans__serif, "sans serif");
 
-  staticpro (&freetype_font_cache);
-  freetype_font_cache = list1 (Qt);
+  staticpro (&freetype_font_cache, list1 (Qt));
 
-  staticpro (&ftfont_generic_family_list);
-  ftfont_generic_family_list = list3 (Fcons (Qmonospace, Qt),
-				      Fcons (Qsans_serif, Qt),
-				      Fcons (Qsans, Qt));
+  staticpro (&ftfont_generic_family_list, list3 (Fcons (Qmonospace, Qt),
+                                                 Fcons (Qsans_serif, Qt),
+                                                 Fcons (Qsans, Qt)));
 
-  staticpro (&ft_face_cache);
-  ft_face_cache = Qnil;
+  staticpro (&ft_face_cache, Qnil);
 
   register_font_driver (&ftfont_driver, NULL);
 }
diff --git a/src/gfilenotify.c b/src/gfilenotify.c
index 650df0fcbb5..63bfdb49460 100644
--- a/src/gfilenotify.c
+++ b/src/gfilenotify.c
@@ -331,7 +331,7 @@ syms_of_gfilenotify (void)
   DEFSYM (Qunmounted, "unmounted");	/* G_FILE_MONITOR_EVENT_UNMOUNTED  */
   DEFSYM (Qmoved, "moved");	/* G_FILE_MONITOR_EVENT_MOVED  */
 
-  staticpro (&watch_list);
+  staticpro (&watch_list, Qnil);
 
   Fprovide (intern_c_string ("gfilenotify"), Qnil);
 
diff --git a/src/inotify.c b/src/inotify.c
index e06cc97c6a7..b10d218b4db 100644
--- a/src/inotify.c
+++ b/src/inotify.c
@@ -546,7 +546,7 @@ syms_of_inotify (void)
   defsubr (&Sinotify_watch_list);
   defsubr (&Sinotify_allocated_p);
 #endif
-  staticpro (&watch_list);
+  staticpro (&watch_list, Qnil);
 
   Fprovide (intern_c_string ("inotify"), Qnil);
 }
diff --git a/src/insdel.c b/src/insdel.c
index 02e3f41bc9f..e9279f0a166 100644
--- a/src/insdel.c
+++ b/src/insdel.c
@@ -2328,10 +2328,8 @@ DEFUN ("combine-after-change-execute", Fcombine_after_change_execute,
 void
 syms_of_insdel (void)
 {
-  staticpro (&combine_after_change_list);
-  staticpro (&combine_after_change_buffer);
-  combine_after_change_list = Qnil;
-  combine_after_change_buffer = Qnil;
+  staticpro (&combine_after_change_list, Qnil);
+  staticpro (&combine_after_change_buffer, Qnil);
 
   DEFSYM (Qundo_auto__undoable_change, "undo-auto--undoable-change");
 
diff --git a/src/keyboard.c b/src/keyboard.c
index c0c863f0d3e..a3a61ed0c63 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -11051,11 +11051,9 @@ static const struct event_head head_table[] = {
 void
 syms_of_keyboard (void)
 {
-  pending_funcalls = Qnil;
-  staticpro (&pending_funcalls);
+  staticpro (&pending_funcalls, Qnil);
 
-  Vlispy_mouse_stem = build_pure_c_string ("mouse");
-  staticpro (&Vlispy_mouse_stem);
+  staticpro (&Vlispy_mouse_stem, build_pure_c_string ("mouse"));
 
   regular_top_level_message = build_pure_c_string ("Back to top level");
 #ifdef HAVE_STACK_OVERFLOW_HANDLING
@@ -11071,13 +11069,10 @@ syms_of_keyboard (void)
   DEFSYM (Qhelp_echo, "help-echo");
   DEFSYM (QCrtl, ":rtl");
 
-  staticpro (&item_properties);
-  item_properties = Qnil;
+  staticpro (&item_properties, Qnil);
 
-  staticpro (&tool_bar_item_properties);
-  tool_bar_item_properties = Qnil;
-  staticpro (&tool_bar_items_vector);
-  tool_bar_items_vector = Qnil;
+  staticpro (&tool_bar_item_properties, Qnil);
+  staticpro (&tool_bar_items_vector, Qnil);
 
   DEFSYM (Qtimer_event_handler, "timer-event-handler");
 
@@ -11242,65 +11237,48 @@ syms_of_keyboard (void)
       }
   }
 
-  button_down_location = Fmake_vector (make_number (5), Qnil);
-  staticpro (&button_down_location);
-  mouse_syms = Fmake_vector (make_number (5), Qnil);
-  staticpro (&mouse_syms);
-  wheel_syms = Fmake_vector (make_number (ARRAYELTS (lispy_wheel_names)),
-			     Qnil);
-  staticpro (&wheel_syms);
+  staticpro (&button_down_location, Fmake_vector (make_number (5), Qnil));
+  staticpro (&mouse_syms, Fmake_vector (make_number (5), Qnil));
+  staticpro (&wheel_syms, Fmake_vector (make_number (ARRAYELTS (lispy_wheel_names)),
+                                        Qnil));
 
   {
     int i;
     int len = ARRAYELTS (modifier_names);
 
-    modifier_symbols = Fmake_vector (make_number (len), Qnil);
+    staticpro (&modifier_symbols, Fmake_vector (make_number (len), Qnil));
     for (i = 0; i < len; i++)
       if (modifier_names[i])
 	ASET (modifier_symbols, i, intern_c_string (modifier_names[i]));
-    staticpro (&modifier_symbols);
   }
 
-  recent_keys = Fmake_vector (make_number (NUM_RECENT_KEYS), Qnil);
-  staticpro (&recent_keys);
+  staticpro (&recent_keys, Fmake_vector (make_number (NUM_RECENT_KEYS), Qnil));
 
-  this_command_keys = Fmake_vector (make_number (40), Qnil);
-  staticpro (&this_command_keys);
+  staticpro (&this_command_keys, Fmake_vector (make_number (40), Qnil));
 
-  raw_keybuf = Fmake_vector (make_number (30), Qnil);
-  staticpro (&raw_keybuf);
+  staticpro (&raw_keybuf, Fmake_vector (make_number (30), Qnil));
 
   DEFSYM (Qcommand_execute, "command-execute");
   DEFSYM (Qinternal_echo_keystrokes_prefix, "internal-echo-keystrokes-prefix");
 
-  accent_key_syms = Qnil;
-  staticpro (&accent_key_syms);
+  staticpro (&accent_key_syms, Qnil);
 
-  func_key_syms = Qnil;
-  staticpro (&func_key_syms);
+  staticpro (&func_key_syms, Qnil);
 
-  drag_n_drop_syms = Qnil;
-  staticpro (&drag_n_drop_syms);
+  staticpro (&drag_n_drop_syms, Qnil);
 
-  unread_switch_frame = Qnil;
-  staticpro (&unread_switch_frame);
+  staticpro (&unread_switch_frame, Qnil);
 
-  internal_last_event_frame = Qnil;
-  staticpro (&internal_last_event_frame);
+  staticpro (&internal_last_event_frame, Qnil);
 
-  read_key_sequence_cmd = Qnil;
-  staticpro (&read_key_sequence_cmd);
-  read_key_sequence_remapped = Qnil;
-  staticpro (&read_key_sequence_remapped);
+  staticpro (&read_key_sequence_cmd, Qnil);
+  staticpro (&read_key_sequence_remapped, Qnil);
 
-  menu_bar_one_keymap_changed_items = Qnil;
-  staticpro (&menu_bar_one_keymap_changed_items);
+  staticpro (&menu_bar_one_keymap_changed_items, Qnil);
 
-  menu_bar_items_vector = Qnil;
-  staticpro (&menu_bar_items_vector);
+  staticpro (&menu_bar_items_vector, Qnil);
 
-  help_form_saved_window_configs = Qnil;
-  staticpro (&help_form_saved_window_configs);
+  staticpro (&help_form_saved_window_configs, Qnil);
 
   defsubr (&Scurrent_idle_time);
   defsubr (&Sevent_symbol_parse_modifiers);
diff --git a/src/keymap.c b/src/keymap.c
index c8cc933e782..7e8b548317e 100644
--- a/src/keymap.c
+++ b/src/keymap.c
@@ -3599,10 +3599,8 @@ void
 syms_of_keymap (void)
 {
   DEFSYM (Qkeymap, "keymap");
-  staticpro (&apropos_predicate);
-  staticpro (&apropos_accumulate);
-  apropos_predicate = Qnil;
-  apropos_accumulate = Qnil;
+  staticpro (&apropos_predicate, Qnil);
+  staticpro (&apropos_accumulate, Qnil);
 
   DEFSYM (Qkeymap_canonicalize, "keymap-canonicalize");
 
@@ -3614,13 +3612,10 @@ syms_of_keymap (void)
      Each one is the value of a Lisp variable, and is also
      pointed to by a C variable */
 
-  global_map = Fmake_keymap (Qnil);
+  staticpro (&global_map, Fmake_keymap (Qnil));
+  staticpro (&current_global_map, global_map);
   Fset (intern_c_string ("global-map"), global_map);
 
-  current_global_map = global_map;
-  staticpro (&global_map);
-  staticpro (&current_global_map);
-
   meta_map = Fmake_keymap (Qnil);
   Fset (intern_c_string ("esc-map"), meta_map);
   Ffset (intern_c_string ("ESC-prefix"), meta_map);
@@ -3629,13 +3624,13 @@ syms_of_keymap (void)
   Fset (intern_c_string ("ctl-x-map"), control_x_map);
   Ffset (intern_c_string ("Control-X-prefix"), control_x_map);
 
-  exclude_keys = listn (CONSTYPE_PURE, 5,
-			pure_cons (build_pure_c_string ("DEL"), build_pure_c_string ("\\d")),
-			pure_cons (build_pure_c_string ("TAB"), build_pure_c_string ("\\t")),
-			pure_cons (build_pure_c_string ("RET"), build_pure_c_string ("\\r")),
-			pure_cons (build_pure_c_string ("ESC"), build_pure_c_string ("\\e")),
-			pure_cons (build_pure_c_string ("SPC"), build_pure_c_string (" ")));
-  staticpro (&exclude_keys);
+  staticpro (&exclude_keys,
+             listn (CONSTYPE_PURE, 5,
+                    pure_cons (build_pure_c_string ("DEL"), build_pure_c_string ("\\d")),
+                    pure_cons (build_pure_c_string ("TAB"), build_pure_c_string ("\\t")),
+                    pure_cons (build_pure_c_string ("RET"), build_pure_c_string ("\\r")),
+                    pure_cons (build_pure_c_string ("ESC"), build_pure_c_string ("\\e")),
+                    pure_cons (build_pure_c_string ("SPC"), build_pure_c_string (" "))));
 
   DEFVAR_LISP ("define-key-rebound-commands", Vdefine_key_rebound_commands,
 	       doc: /* List of commands given new key bindings recently.
@@ -3689,17 +3684,16 @@ be preferred.  */);
   DEFSYM (Qmenu_bar, "menu-bar");
   DEFSYM (Qmode_line, "mode-line");
 
-  staticpro (&Vmouse_events);
-  Vmouse_events = listn (CONSTYPE_PURE, 9,
-			 Qmenu_bar,
-			 Qtool_bar,
-			 Qheader_line,
-			 Qmode_line,
-			 intern_c_string ("mouse-1"),
-			 intern_c_string ("mouse-2"),
-			 intern_c_string ("mouse-3"),
-			 intern_c_string ("mouse-4"),
-			 intern_c_string ("mouse-5"));
+  staticpro (&Vmouse_events, listn (CONSTYPE_PURE, 9,
+                                    Qmenu_bar,
+                                    Qtool_bar,
+                                    Qheader_line,
+                                    Qmode_line,
+                                    intern_c_string ("mouse-1"),
+                                    intern_c_string ("mouse-2"),
+                                    intern_c_string ("mouse-3"),
+                                    intern_c_string ("mouse-4"),
+                                    intern_c_string ("mouse-5")));
 
   /* Keymap used for minibuffers when doing completion.  */
   /* Keymap used for minibuffers when doing completion and require a match.  */
@@ -3709,13 +3703,11 @@ be preferred.  */);
   DEFSYM (Qremap, "remap");
   DEFSYM (QCadvertised_binding, ":advertised-binding");
 
-  command_remapping_vector = Fmake_vector (make_number (2), Qremap);
-  staticpro (&command_remapping_vector);
+  staticpro (&command_remapping_vector,
+             Fmake_vector (make_number (2), Qremap));
 
-  where_is_cache_keymaps = Qt;
-  where_is_cache = Qnil;
-  staticpro (&where_is_cache);
-  staticpro (&where_is_cache_keymaps);
+  staticpro (&where_is_cache, Qnil);
+  staticpro (&where_is_cache_keymaps, Qt);
 
   defsubr (&Skeymapp);
   defsubr (&Skeymap_parent);
diff --git a/src/kqueue.c b/src/kqueue.c
index 7a4f6a471c4..185e0e35820 100644
--- a/src/kqueue.c
+++ b/src/kqueue.c
@@ -527,7 +527,7 @@ syms_of_kqueue (void)
   DEFSYM (Qlink, "link");	/* NOTE_LINK  */
   DEFSYM (Qrename, "rename");	/* NOTE_RENAME  */
 
-  staticpro (&watch_list);
+  staticpro (&watch_list, Qnil);
 
   Fprovide (intern_c_string ("kqueue"), Qnil);
 }
diff --git a/src/lisp.h b/src/lisp.h
index b931d23bf38..19a8a8027e3 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3281,7 +3281,23 @@ extern Lisp_Object Vascii_canon_table;
 \f
 /* Call staticpro (&var) to protect static variable `var'.  */
 
-void staticpro (Lisp_Object *);
+void staticpro_1 (Lisp_Object *);
+
+INLINE void
+staticpro (Lisp_Object *ptr, Lisp_Object initial_value)
+{
+  eassume (*ptr == Qnil);
+#if 4 < __GNUC__
+  if (__builtin_constant_p(initial_value == Qnil) &&
+      initial_value == Qnil)
+    ;
+  else
+#endif
+    *ptr = initial_value;
+
+  staticpro_1 (ptr);
+}
+
 \f
 /* Forward declarations for prototypes.  */
 struct window;
diff --git a/src/lread.c b/src/lread.c
index 8fb61f56338..e63d7431e46 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -4344,8 +4344,7 @@ void
 init_obarray (void)
 {
   Vobarray = Fmake_vector (make_number (OBARRAY_SIZE), make_number (0));
-  initial_obarray = Vobarray;
-  staticpro (&initial_obarray);
+  staticpro (&initial_obarray, Vobarray);
 
   for (int i = 0; i < ARRAYELTS (lispsym); i++)
     define_symbol (builtin_lisp_symbol (i), defsym_name[i]);
@@ -4443,7 +4442,7 @@ defvar_lisp (struct Lisp_Objfwd *o_fwd,
 	     const char *namestring, Lisp_Object *address)
 {
   defvar_lisp_nopro (o_fwd, namestring, address);
-  staticpro (address);
+  staticpro (address, Qnil);
 }
 
 /* Similar but define a variable whose value is the Lisp Object stored
@@ -5089,13 +5088,10 @@ this variable will become obsolete.  */);
   DEFSYM (Qdir_ok, "dir-ok");
   DEFSYM (Qdo_after_load_evaluation, "do-after-load-evaluation");
 
-  staticpro (&read_objects_map);
-  read_objects_map = Qnil;
-  staticpro (&read_objects_completed);
-  read_objects_completed = Qnil;
+  staticpro (&read_objects_map, Qnil);
+  staticpro (&read_objects_completed, Qnil);
 
-  Vloads_in_progress = Qnil;
-  staticpro (&Vloads_in_progress);
+  staticpro (&Vloads_in_progress, Qnil);
 
   DEFSYM (Qhash_table, "hash-table");
   DEFSYM (Qdata, "data");
diff --git a/src/macfont.m b/src/macfont.m
index 817071fa44f..5953ce7fd23 100644
--- a/src/macfont.m
+++ b/src/macfont.m
@@ -4050,6 +4050,5 @@ So we use CTFontDescriptorCreateMatchingFontDescriptor (no
   /* The boolean-valued font property key specifying the use of leading.  */
   DEFSYM (QCminspace, ":minspace");
 
-  macfont_family_cache = Qnil;
-  staticpro (&macfont_family_cache);
+  staticpro (&macfont_family_cache, Qnil);
 }
diff --git a/src/menu.c b/src/menu.c
index 93e793a5d91..3be8ecd5fab 100644
--- a/src/menu.c
+++ b/src/menu.c
@@ -1580,8 +1580,7 @@ for instance using the window manager, then this produces a quit and
 void
 syms_of_menu (void)
 {
-  staticpro (&menu_items);
-  menu_items = Qnil;
+  staticpro (&menu_items, Qnil);
   menu_items_inuse = Qnil;
 
   defsubr (&Sx_popup_menu);
diff --git a/src/minibuf.c b/src/minibuf.c
index 45cf15224a5..11c6b94ae5d 100644
--- a/src/minibuf.c
+++ b/src/minibuf.c
@@ -1908,19 +1908,16 @@ If no minibuffer is active, return nil.  */)
 void
 init_minibuf_once (void)
 {
-  Vminibuffer_list = Qnil;
-  staticpro (&Vminibuffer_list);
+  staticpro (&Vminibuffer_list, Qnil);
 }
 
 void
 syms_of_minibuf (void)
 {
   minibuf_level = 0;
-  minibuf_prompt = Qnil;
-  staticpro (&minibuf_prompt);
+  staticpro (&minibuf_prompt, Qnil);
 
-  minibuf_save_list = Qnil;
-  staticpro (&minibuf_save_list);
+  staticpro (&minibuf_save_list, Qnil);
 
   DEFSYM (Qcompletion_ignore_case, "completion-ignore-case");
   DEFSYM (Qminibuffer_default, "minibuffer-default");
@@ -1928,8 +1925,7 @@ syms_of_minibuf (void)
 
   DEFSYM (Qminibuffer_completion_table, "minibuffer-completion-table");
 
-  staticpro (&last_minibuf_string);
-  last_minibuf_string = Qnil;
+  staticpro (&last_minibuf_string, Qnil);
 
   DEFSYM (Qminibuffer_history, "minibuffer-history");
   DEFSYM (Qbuffer_name_history, "buffer-name-history");
diff --git a/src/msdos.c b/src/msdos.c
index eedbf7b1a6c..7ca6d03917a 100644
--- a/src/msdos.c
+++ b/src/msdos.c
@@ -4196,8 +4196,7 @@ msdos_fatal_signal (int sig)
 void
 syms_of_msdos (void)
 {
-  recent_doskeys = Fmake_vector (make_number (NUM_RECENT_DOSKEYS), Qnil);
-  staticpro (&recent_doskeys);
+  staticpro (&recent_doskeys, Fmake_vector (make_number (NUM_RECENT_DOSKEYS), Qnil));
 
 #ifndef HAVE_X_WINDOWS
 
diff --git a/src/nsselect.m b/src/nsselect.m
index c72f179ab38..6d0b6342dc2 100644
--- a/src/nsselect.m
+++ b/src/nsselect.m
@@ -492,8 +492,7 @@ Updated by Christian Limpach (chris@nice.ch)
   defsubr (&Sns_selection_exists_p);
   defsubr (&Sns_selection_owner_p);
 
-  Vselection_alist = Qnil;
-  staticpro (&Vselection_alist);
+  staticpro (&Vselection_alist, Qnil);
 
   DEFVAR_LISP ("ns-sent-selection-hooks", Vns_sent_selection_hooks,
                "A list of functions to be called when Emacs answers a selection request.\n\
diff --git a/src/print.c b/src/print.c
index a8bbb9d37a1..6867981102b 100644
--- a/src/print.c
+++ b/src/print.c
@@ -2435,7 +2435,7 @@ priorities.  */);
   Vprint_charset_text_property = Qdefault;
 
   /* prin1_to_string_buffer initialized in init_buffer_once in buffer.c */
-  staticpro (&Vprin1_to_string_buffer);
+  staticpro (&Vprin1_to_string_buffer, Qnil);
 
   defsubr (&Sprin1);
   defsubr (&Sprin1_to_string);
@@ -2452,6 +2452,5 @@ priorities.  */);
   DEFSYM (Qprint_escape_nonascii, "print-escape-nonascii");
   DEFSYM (Qprint_escape_control_characters, "print-escape-control-characters");
 
-  print_prune_charset_plist = Qnil;
-  staticpro (&print_prune_charset_plist);
+  staticpro (&print_prune_charset_plist, Qnil);
 }
diff --git a/src/process.c b/src/process.c
index ed2cab7b51f..25634f753e3 100644
--- a/src/process.c
+++ b/src/process.c
@@ -8149,8 +8149,8 @@ syms_of_process (void)
 
   DEFSYM (Qlast_nonmenu_event, "last-nonmenu-event");
 
-  staticpro (&Vprocess_alist);
-  staticpro (&deleted_pid_list);
+  staticpro (&Vprocess_alist, Qnil);
+  staticpro (&deleted_pid_list, Qnil);
 
 #endif	/* subprocesses */
 
diff --git a/src/profiler.c b/src/profiler.c
index 312574d7526..b1e54b6a748 100644
--- a/src/profiler.c
+++ b/src/profiler.c
@@ -595,16 +595,14 @@ to make room for new entries.  */);
 
 #ifdef PROFILER_CPU_SUPPORT
   profiler_cpu_running = NOT_RUNNING;
-  cpu_log = Qnil;
-  staticpro (&cpu_log);
+  staticpro (&cpu_log, Qnil);
   defsubr (&Sprofiler_cpu_start);
   defsubr (&Sprofiler_cpu_stop);
   defsubr (&Sprofiler_cpu_running_p);
   defsubr (&Sprofiler_cpu_log);
 #endif
   profiler_memory_running = false;
-  memory_log = Qnil;
-  staticpro (&memory_log);
+  staticpro (&memory_log, Qnil);
   defsubr (&Sprofiler_memory_start);
   defsubr (&Sprofiler_memory_stop);
   defsubr (&Sprofiler_memory_running_p);
diff --git a/src/search.c b/src/search.c
index 842e9309a2e..aa4e39bdbbc 100644
--- a/src/search.c
+++ b/src/search.c
@@ -3367,12 +3367,9 @@ syms_of_search (void)
       searchbufs[i].buf.allocated = 100;
       searchbufs[i].buf.buffer = xmalloc (100);
       searchbufs[i].buf.fastmap = searchbufs[i].fastmap;
-      searchbufs[i].regexp = Qnil;
-      searchbufs[i].f_whitespace_regexp = Qnil;
-      searchbufs[i].syntax_table = Qnil;
-      staticpro (&searchbufs[i].regexp);
-      staticpro (&searchbufs[i].f_whitespace_regexp);
-      staticpro (&searchbufs[i].syntax_table);
+      staticpro (&searchbufs[i].regexp, Qnil);
+      staticpro (&searchbufs[i].f_whitespace_regexp, Qnil);
+      staticpro (&searchbufs[i].syntax_table, Qnil);
       searchbufs[i].next = (i == REGEXP_CACHE_SIZE-1 ? 0 : &searchbufs[i+1]);
     }
   searchbuf_head = &searchbufs[0];
@@ -3403,11 +3400,9 @@ syms_of_search (void)
   Fput (Qinvalid_regexp, Qerror_message,
 	build_pure_c_string ("Invalid regexp"));
 
-  last_thing_searched = Qnil;
-  staticpro (&last_thing_searched);
+  staticpro (&last_thing_searched, Qnil);
 
-  saved_last_thing_searched = Qnil;
-  staticpro (&saved_last_thing_searched);
+  staticpro (&saved_last_thing_searched, Qnil);
 
   DEFVAR_LISP ("search-spaces-regexp", Vsearch_spaces_regexp,
       doc: /* Regexp to substitute for bunches of spaces in regexp search.
diff --git a/src/syntax.c b/src/syntax.c
index c5a4b03955b..3ab82ad5d64 100644
--- a/src/syntax.c
+++ b/src/syntax.c
@@ -3722,15 +3722,15 @@ syms_of_syntax (void)
 	       doc: /* Non-nil means `forward-comment' can use `syntax-ppss' internally.  */);
   Vcomment_use_syntax_ppss = Qt;
 
-  staticpro (&Vsyntax_code_object);
+  staticpro (&Vsyntax_code_object, Qnil);
 
-  staticpro (&gl_state.object);
-  staticpro (&gl_state.global_code);
-  staticpro (&gl_state.current_syntax_table);
-  staticpro (&gl_state.old_prop);
+  staticpro (&gl_state.object, Qnil);
+  staticpro (&gl_state.global_code, Qnil);
+  staticpro (&gl_state.current_syntax_table, Qnil);
+  staticpro (&gl_state.old_prop, Qnil);
 
   /* Defined in regex.c.  */
-  staticpro (&re_match_object);
+  staticpro (&re_match_object, Qnil);
 
   DEFSYM (Qscan_error, "scan-error");
   Fput (Qscan_error, Qerror_conditions,
diff --git a/src/textprop.c b/src/textprop.c
index 984f2e66406..7f7d7359ffc 100644
--- a/src/textprop.c
+++ b/src/textprop.c
@@ -2340,10 +2340,8 @@ inherits it if NONSTICKINESS is nil.  The `front-sticky' and
   Vtext_property_default_nonsticky
     = list2 (Fcons (Qsyntax_table, Qt), Fcons (Qdisplay, Qt));
 
-  staticpro (&interval_insert_behind_hooks);
-  staticpro (&interval_insert_in_front_hooks);
-  interval_insert_behind_hooks = Qnil;
-  interval_insert_in_front_hooks = Qnil;
+  staticpro (&interval_insert_behind_hooks, Qnil);
+  staticpro (&interval_insert_in_front_hooks, Qnil);
 
 
   /* Common attributes one might give text.  */
diff --git a/src/thread.c b/src/thread.c
index f11e3e5addb..fe0c8688415 100644
--- a/src/thread.c
+++ b/src/thread.c
@@ -1074,8 +1074,7 @@ syms_of_threads (void)
       defsubr (&Scondition_name);
       defsubr (&Sthread_last_error);
 
-      staticpro (&last_thread_error);
-      last_thread_error = Qnil;
+      staticpro (&last_thread_error, Qnil);
     }
 
   DEFSYM (Qthreadp, "threadp");
diff --git a/src/undo.c b/src/undo.c
index c34faa42720..a34dd486094 100644
--- a/src/undo.c
+++ b/src/undo.c
@@ -430,8 +430,7 @@ syms_of_undo (void)
   /* Marker for function call undo list elements.  */
   DEFSYM (Qapply, "apply");
 
-  pending_boundary = Qnil;
-  staticpro (&pending_boundary);
+  staticpro (&pending_boundary, Qnil);
 
   defsubr (&Sundo_boundary);
 
diff --git a/src/w32fns.c b/src/w32fns.c
index 2b920f29c65..5903b2a1557 100644
--- a/src/w32fns.c
+++ b/src/w32fns.c
@@ -10164,8 +10164,7 @@ syms_of_w32fns (void)
   Fput (Qundefined_color, Qerror_message,
 	build_pure_c_string ("Undefined color"));
 
-  staticpro (&w32_grabbed_keys);
-  w32_grabbed_keys = Qnil;
+  staticpro (&w32_grabbed_keys, Qnil);
 
   DEFVAR_LISP ("w32-color-map", Vw32_color_map,
 	       doc: /* An array of color name mappings for Windows.  */);
@@ -10515,16 +10514,11 @@ tip frame.  */);
   defsubr (&Sset_message_beep);
   defsubr (&Sx_show_tip);
   defsubr (&Sx_hide_tip);
-  tip_timer = Qnil;
-  staticpro (&tip_timer);
-  tip_frame = Qnil;
-  staticpro (&tip_frame);
-  tip_last_frame = Qnil;
-  staticpro (&tip_last_frame);
-  tip_last_string = Qnil;
-  staticpro (&tip_last_string);
-  tip_last_parms = Qnil;
-  staticpro (&tip_last_parms);
+  staticpro (&tip_timer, Qnil);
+  staticpro (&tip_frame, Qnil);
+  staticpro (&tip_last_frame, Qnil);
+  staticpro (&tip_last_string, Qnil);
+  staticpro (&tip_last_parms, Qnil);
 
   defsubr (&Sx_file_dialog);
 #ifdef WINDOWSNT
diff --git a/src/w32proc.c b/src/w32proc.c
index 28d7b6611f6..83b8ef7f56d 100644
--- a/src/w32proc.c
+++ b/src/w32proc.c
@@ -3826,7 +3826,7 @@ in the strings.  So do NOT use this option when comparing file names
 for equality, only when you need to sort them.  */);
   Vw32_collate_ignore_punctuation = Qnil;
 
-  staticpro (&Vw32_valid_locale_ids);
-  staticpro (&Vw32_valid_codepages);
+  staticpro (&Vw32_valid_locale_ids, Qnil);
+  staticpro (&Vw32_valid_codepages, Qnil);
 }
 /* end of w32proc.c */
diff --git a/src/w32select.c b/src/w32select.c
index a9df3f770b7..cd1bf6a60cb 100644
--- a/src/w32select.c
+++ b/src/w32select.c
@@ -1181,13 +1181,13 @@ syms_of_w32select (void)
 
   DEFSYM (QCLIPBOARD, "CLIPBOARD");
 
-  cfg_coding_system = Qnil;     staticpro (&cfg_coding_system);
-  current_text = Qnil;		staticpro (&current_text);
-  current_coding_system = Qnil; staticpro (&current_coding_system);
+  staticpro (&cfg_coding_system, Qnil);
+  staticpro (&current_text, Qnil);
+  staticpro (&current_coding_system, Qnil);
 
   DEFSYM (Qutf_16le_dos, "utf-16le-dos");
-  QANSICP = Qnil; staticpro (&QANSICP);
-  QOEMCP = Qnil;  staticpro (&QOEMCP);
+  staticpro (&QANSICP, Qnil);
+  staticpro (&QOEMCP, Qnil);
 }
 
 /* One-time init.  Called in the dumped Emacs, but not in the
diff --git a/src/window.c b/src/window.c
index e6d0280d9b0..581d529bf8c 100644
--- a/src/window.c
+++ b/src/window.c
@@ -7566,10 +7566,8 @@ syms_of_window (void)
   DEFSYM (Qmode_line_format, "mode-line-format");
   DEFSYM (Qheader_line_format, "header-line-format");
 
-  staticpro (&Vwindow_list);
-
-  minibuf_selected_window = Qnil;
-  staticpro (&minibuf_selected_window);
+  staticpro (&Vwindow_list, Qnil);
+  staticpro (&minibuf_selected_window, Qnil);
 
   window_scroll_pixel_based_preserve_x = -1;
   window_scroll_pixel_based_preserve_y = -1;
diff --git a/src/xdisp.c b/src/xdisp.c
index df5335e4acc..e194f767dae 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -32418,11 +32418,9 @@ x_intersect_rectangles (XRectangle *r1, XRectangle *r2, XRectangle *result)
 void
 syms_of_xdisp (void)
 {
-  Vwith_echo_area_save_vector = Qnil;
-  staticpro (&Vwith_echo_area_save_vector);
+  staticpro (&Vwith_echo_area_save_vector, Qnil);
 
-  Vmessage_stack = Qnil;
-  staticpro (&Vmessage_stack);
+  staticpro (&Vmessage_stack, Qnil);
 
   /* Non-nil means don't actually do any redisplay.  */
   DEFSYM (Qinhibit_redisplay, "inhibit-redisplay");
@@ -32434,12 +32432,9 @@ syms_of_xdisp (void)
 They are still logged to the *Messages* buffer.  */);
   inhibit_message = 0;
 
-  message_dolog_marker1 = Fmake_marker ();
-  staticpro (&message_dolog_marker1);
-  message_dolog_marker2 = Fmake_marker ();
-  staticpro (&message_dolog_marker2);
-  message_dolog_marker3 = Fmake_marker ();
-  staticpro (&message_dolog_marker3);
+  staticpro (&message_dolog_marker1, Fmake_marker ());
+  staticpro (&message_dolog_marker2, Fmake_marker ());
+  staticpro (&message_dolog_marker3, Fmake_marker ());
 
   defsubr (&Sset_buffer_redisplay);
 #ifdef GLYPH_DEBUG
@@ -32553,8 +32548,7 @@ They are still logged to the *Messages* buffer.  */);
 
   DEFSYM (Qinhibit_free_realized_faces, "inhibit-free-realized-faces");
 
-  list_of_error = list1 (list2 (Qerror, Qvoid_variable));
-  staticpro (&list_of_error);
+  staticpro (&list_of_error, list1 (list2 (Qerror, Qvoid_variable)));
 
   /* Values of those variables at last redisplay are stored as
      properties on 'overlay-arrow-position' symbol.  However, if
@@ -32568,38 +32562,26 @@ They are still logged to the *Messages* buffer.  */);
   DEFSYM (Qoverlay_arrow_string, "overlay-arrow-string");
   DEFSYM (Qoverlay_arrow_bitmap, "overlay-arrow-bitmap");
 
-  echo_buffer[0] = echo_buffer[1] = Qnil;
-  staticpro (&echo_buffer[0]);
-  staticpro (&echo_buffer[1]);
+  staticpro (&echo_buffer[0], Qnil);
+  staticpro (&echo_buffer[1], Qnil);
 
-  echo_area_buffer[0] = echo_area_buffer[1] = Qnil;
-  staticpro (&echo_area_buffer[0]);
-  staticpro (&echo_area_buffer[1]);
+  staticpro (&echo_area_buffer[0], Qnil);
+  staticpro (&echo_area_buffer[1], Qnil);
 
-  Vmessages_buffer_name = build_pure_c_string ("*Messages*");
-  staticpro (&Vmessages_buffer_name);
+  staticpro (&Vmessages_buffer_name, build_pure_c_string ("*Messages*"));
 
-  mode_line_proptrans_alist = Qnil;
-  staticpro (&mode_line_proptrans_alist);
-  mode_line_string_list = Qnil;
-  staticpro (&mode_line_string_list);
-  mode_line_string_face = Qnil;
-  staticpro (&mode_line_string_face);
-  mode_line_string_face_prop = Qnil;
-  staticpro (&mode_line_string_face_prop);
-  Vmode_line_unwind_vector = Qnil;
-  staticpro (&Vmode_line_unwind_vector);
+  staticpro (&mode_line_proptrans_alist, Qnil);
+  staticpro (&mode_line_string_list, Qnil);
+  staticpro (&mode_line_string_face, Qnil);
+  staticpro (&mode_line_string_face_prop, Qnil);
+  staticpro (&Vmode_line_unwind_vector, Qnil);
 
   DEFSYM (Qmode_line_default_help_echo, "mode-line-default-help-echo");
 
-  help_echo_string = Qnil;
-  staticpro (&help_echo_string);
-  help_echo_object = Qnil;
-  staticpro (&help_echo_object);
-  help_echo_window = Qnil;
-  staticpro (&help_echo_window);
-  previous_help_echo_string = Qnil;
-  staticpro (&previous_help_echo_string);
+  staticpro (&help_echo_string, Qnil);
+  staticpro (&help_echo_object, Qnil);
+  staticpro (&help_echo_window, Qnil);
+  staticpro (&previous_help_echo_string, Qnil);
   help_echo_pos = -1;
 
   DEFSYM (Qright_to_left, "right-to-left");
diff --git a/src/xfaces.c b/src/xfaces.c
index a9c2f37e9f2..89c1065750e 100644
--- a/src/xfaces.c
+++ b/src/xfaces.c
@@ -6451,12 +6451,9 @@ syms_of_xfaces (void)
   /* The name of the function used to compute colors on TTYs.  */
   DEFSYM (Qtty_color_alist, "tty-color-alist");
 
-  Vparam_value_alist = list1 (Fcons (Qnil, Qnil));
-  staticpro (&Vparam_value_alist);
-  Vface_alternative_font_family_alist = Qnil;
-  staticpro (&Vface_alternative_font_family_alist);
-  Vface_alternative_font_registry_alist = Qnil;
-  staticpro (&Vface_alternative_font_registry_alist);
+  staticpro (&Vparam_value_alist, list1 (Fcons (Qnil, Qnil)));
+  staticpro (&Vface_alternative_font_family_alist, Qnil);
+  staticpro (&Vface_alternative_font_registry_alist, Qnil);
 
   defsubr (&Sinternal_make_lisp_face);
   defsubr (&Sinternal_lisp_face_p);
diff --git a/src/xfns.c b/src/xfns.c
index 78151c81380..38d9f91dc89 100644
--- a/src/xfns.c
+++ b/src/xfns.c
@@ -7926,16 +7926,11 @@ When using Gtk+ tooltips, the tooltip face is not used.  */);
   defsubr (&Sx_show_tip);
   defsubr (&Sx_hide_tip);
   defsubr (&Sx_double_buffered_p);
-  tip_timer = Qnil;
-  staticpro (&tip_timer);
-  tip_frame = Qnil;
-  staticpro (&tip_frame);
-  tip_last_frame = Qnil;
-  staticpro (&tip_last_frame);
-  tip_last_string = Qnil;
-  staticpro (&tip_last_string);
-  tip_last_parms = Qnil;
-  staticpro (&tip_last_parms);
+  staticpro (&tip_timer, Qnil);
+  staticpro (&tip_frame, Qnil);
+  staticpro (&tip_last_frame, Qnil);
+  staticpro (&tip_last_string, Qnil);
+  staticpro (&tip_last_parms, Qnil);
 
   defsubr (&Sx_uses_old_gtk_dialog);
 #if defined (USE_MOTIF) || defined (USE_GTK)
diff --git a/src/xfont.c b/src/xfont.c
index c2e416bc058..a67237aa2aa 100644
--- a/src/xfont.c
+++ b/src/xfont.c
@@ -1098,9 +1098,7 @@ struct font_driver const xfont_driver =
 void
 syms_of_xfont (void)
 {
-  staticpro (&xfont_scripts_cache);
-  xfont_scripts_cache = CALLN (Fmake_hash_table, QCtest, Qequal);
-  staticpro (&xfont_scratch_props);
-  xfont_scratch_props = Fmake_vector (make_number (8), Qnil);
+  staticpro (&xfont_scripts_cache, CALLN (Fmake_hash_table, QCtest, Qequal));
+  staticpro (&xfont_scratch_props, Fmake_vector (make_number (8), Qnil));
   register_font_driver (&xfont_driver, NULL);
 }
diff --git a/src/xselect.c b/src/xselect.c
index ecf59df2943..538b555f643 100644
--- a/src/xselect.c
+++ b/src/xselect.c
@@ -2633,15 +2633,13 @@ syms_of_xselect (void)
   defsubr (&Sx_send_client_message);
   defsubr (&Sx_register_dnd_atom);
 
-  reading_selection_reply = Fcons (Qnil, Qnil);
-  staticpro (&reading_selection_reply);
+  staticpro (&reading_selection_reply, Fcons (Qnil, Qnil));
   reading_selection_window = 0;
   reading_which_selection = 0;
 
   property_change_wait_list = 0;
   prop_location_identifier = 0;
-  property_change_reply = Fcons (Qnil, Qnil);
-  staticpro (&property_change_reply);
+  staticpro (&property_change_reply, Fcons (Qnil, Qnil));
 
   converted_selections = NULL;
   conversion_fail_tag = None;
diff --git a/src/xterm.c b/src/xterm.c
index 6ab4a03002d..d1c3797cf06 100644
--- a/src/xterm.c
+++ b/src/xterm.c
@@ -13257,8 +13257,7 @@ syms_of_xterm (void)
   DEFSYM (Qlatin_1, "latin-1");
 
 #ifdef USE_GTK
-  xg_default_icon_file = build_pure_c_string ("icons/hicolor/scalable/apps/emacs.svg");
-  staticpro (&xg_default_icon_file);
+  staticpro (&xg_default_icon_file, build_pure_c_string ("icons/hicolor/scalable/apps/emacs.svg"));
 
   DEFSYM (Qx_gtk_map_stock, "x-gtk-map-stock");
 #endif

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Daniel Colascione <dancol@dancol.org>
> Date: Thu, 29 Mar 2018 00:34:16 -0700
> 
> I'm not sure what's left to test. It works. What's stopping a merge?

I think what is left is minor.  Some things that come to mind:

  . Some minimal documentation (manual, NEWS).

  . I don't think versioned installs are supported, as the code will
    always try to load emacs.pdmp, AFAIR, even when the binary is
    called emacs-XX.YY.ZZ.

  . There's also the issue of the "legacy" dumping.  I'm guessing we
    will want to leave that in the sources for the time being?  If so,
    did anyone try to build the branch using that, and did it work as
    expected?

  . One other thing to try is to build on one system, then unpack the
    binary distro on another binary-compatible system, perhaps with a
    different directory tree structure, and see if everything works.

Re: Preview: portable dumper

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Colascione <dancol@dancol.org>
>> Date: Thu, 29 Mar 2018 00:34:16 -0700
>> 
>> I'm not sure what's left to test. It works. What's stopping a merge?
>
> I think what is left is minor.  Some things that come to mind:
>
>   . Some minimal documentation (manual, NEWS).

And in configure. Itʼs not clear to me what the various combinations
of --with-dumping and --with-pdumper are supposed to see (see below).

>   . There's also the issue of the "legacy" dumping.  I'm guessing we
>     will want to leave that in the sources for the time being?  If so,
>     did anyone try to build the branch using that, and did it work as
>     expected?

./configure --with-dumping=unexec --with-pdumper=no && src/emacs -Q
# uses unexec, doesnʼt support pdump. OK

./configure --with-dumping=unexec && src/emacs -Q
# uses unexec, doesnʼt support pdump. OK

./configure --with-dumping=unexec --with-pdumper=yes && src/emacs -Q
# uses unexec, supports pdump. Runs OK. What does 'supports pdump'
mean in this context? That we'll try to load a dump file even though
we've been built with unexec?

In any case, the old unexec code looks like it still works on my
system.

Robert

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Daniel Colascione <dancol@dancol.org>,  angelo.g0@libero.it,  emacs-devel@gnu.org
> Date: Thu, 29 Mar 2018 15:03:05 +0200
> 
> In any case, the old unexec code looks like it still works on my
> system.

Thanks for testing it.

Re: Preview: portable dumper

[Angelo Graziosi]

> Il 29 marzo 2018 alle 13.53 Eli Zaretskii ha scritto:
> 
> 
> > From: Daniel Colascione
> > Date: Thu, 29 Mar 2018 00:34:16 -0700
> > 
> > I'm not sure what's left to test. It works. What's stopping a merge?
> 
> I think what is left is minor.  Some things that come to mind:
> 
>  [...]
>   . One other thing to try is to build on one system, then unpack the
>     binary distro on another binary-compatible system, perhaps with a
>     different directory tree structure, and see if everything works.

I build Emacs also for Windows Subsystem for Linux (WSL) but I do the build on GNU/Linux Mint (*) because it is a bit faster. The build creates a tar ball which I then install on WSL (both in Win 10 Pro 64 and in Win 10 Home 64). This build does not use X because WSL doesn't too..

Generally, the build I do on Win 10 64 (W64 native, i.e. MSYS2/MinGW64) is also installed in Win 10 Home.


---
(*) Both Mint and WSL uses Ubuntu 16.04

Re: Preview: portable dumper

[John Wiegley]

>>>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:

EZ> There's also the issue of the "legacy" dumping. I'm guessing we will want
EZ> to leave that in the sources for the time being?

I think we should, but not for much longer than two major release cycles.

John

Re: Preview: portable dumper

[Stefan Monnier]

> EZ> There's also the issue of the "legacy" dumping. I'm guessing we will want
> EZ> to leave that in the sources for the time being?
> I think we should, but not for much longer than two major release cycles.
                                                  ^^^
                                                  one
-- Stefan

Re: Preview: portable dumper

[Alan Third]

On Thu, Mar 29, 2018 at 12:34:16AM -0700, Daniel Colascione wrote:
>
> I'm not sure what's left to test. It works. What's stopping a merge?
> 

I just tried to build on macOS for the first time in a while and I get
this in ‘make install’

    Unknown option ‘--temacs=pbootstrap’
    make[1]: *** [bootstrap-emacs.pdmp] Error 255
    make: *** [src] Error 2
    make: *** Waiting for unfinished jobs....

I also think we still need to get the pdmp file copied into the
Emacs.app file structure.
-- 
Alan Third

Re: Preview: portable dumper

[dancol]

> On Thu, Mar 29, 2018 at 12:34:16AM -0700, Daniel Colascione wrote:
>>
>> I'm not sure what's left to test. It works. What's stopping a merge?
>>
>
> I just tried to build on macOS for the first time in a while and I get
> this in ‘make install’
>
>     Unknown option ‘--temacs=pbootstrap’

Is pdumper enabled in your config.h?

>     make[1]: *** [bootstrap-emacs.pdmp] Error 255
>     make: *** [src] Error 2
>     make: *** Waiting for unfinished jobs....
>
> I also think we still need to get the pdmp file copied into the
> Emacs.app file structure.

Probably.

Re: Preview: portable dumper

[Alan Third]

On Thu, Mar 29, 2018 at 12:32:29PM -0700, dancol@dancol.org wrote:
> > On Thu, Mar 29, 2018 at 12:34:16AM -0700, Daniel Colascione wrote:
> >>
> >> I'm not sure what's left to test. It works. What's stopping a merge?
> >>
> >
> > I just tried to build on macOS for the first time in a while and I get
> > this in ‘make install’
> >
> >     Unknown option ‘--temacs=pbootstrap’
> 
> Is pdumper enabled in your config.h?

Sorry, never mind. I rebuilt from scratch and it’s fine. Must have
been something left over from a previous build.

-- 
Alan Third

Re: Preview: portable dumper

[Alan Third]

[-- Attachment #1: Type: text/plain, Size: 297 bytes --]

On Thu, Mar 29, 2018 at 08:17:57PM +0100, Alan Third wrote:
> I also think we still need to get the pdmp file copied into the
> Emacs.app file structure.

Patch attached. It looks like it works but I don’t know much about
makefiles so I don’t know if it’s the right approach.
-- 
Alan Third

[-- Attachment #2: 0001-Copy-pdmp-file-to-Emacs.app.patch --]
[-- Type: text/plain, Size: 1043 bytes --]

From faaaeceec4a9f199942bd35b1b83e0345c828d82 Mon Sep 17 00:00:00 2001
From: Alan Third <alan@idiocy.org>
Date: Sat, 31 Mar 2018 10:49:59 +0100
Subject: [PATCH] Copy pdmp file to Emacs.app

* nextstep/Makefile.in: Handle emacs.pdmp file.
---
 nextstep/Makefile.in | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/nextstep/Makefile.in b/nextstep/Makefile.in
index cb69898737..696a18b27d 100644
--- a/nextstep/Makefile.in
+++ b/nextstep/Makefile.in
@@ -44,7 +44,7 @@ ns_check_file =
 
 .PHONY: all
 
-all: ${ns_appdir} ${ns_appbindir}/Emacs
+all: ${ns_appdir} ${ns_appbindir}/Emacs ${ns_appbindir}/emacs.pdmp
 
 ${ns_check_file} ${ns_appdir}: ${srcdir}/${ns_appsrc} ${ns_appsrc}
 	rm -rf ${ns_appdir}
@@ -63,6 +63,10 @@ ${ns_appbindir}/Emacs:
 	${MKDIR_P} ${ns_appbindir}
 	cp -f ../src/emacs${EXEEXT} $@
 
+${ns_appbindir}/emacs.pdmp: ${ns_appdir} ${ns_check_file} ../src/emacs${EXEEXT}.pdmp
+	${MKDIR_P} ${ns_appbindir}
+	cp -f ../src/emacs${EXEEXT}.pdmp $@
+
 .PHONY: FORCE
 
 ../src/emacs${EXEEXT}: FORCE
-- 
2.16.1

Re: Preview: portable dumper

[Robert Pluim]

Alan Third <alan@idiocy.org> writes:

> On Thu, Mar 29, 2018 at 08:17:57PM +0100, Alan Third wrote:
>> I also think we still need to get the pdmp file copied into the
>> Emacs.app file structure.
>
> Patch attached. It looks like it works but I don’t know much about
> makefiles so I don’t know if it’s the right approach.

The pdumper branch has not yet been merged to master.

Is there anything holding up the merge? It worked fine last time I
tested it.

Regards

Robert

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Fri, 08 Jun 2018 11:29:29 +0200
> Cc: Alan Third <alan@idiocy.org>, Daniel Colascione <dancol@dancol.org>,
> 	Angelo Graziosi <angelo.g0@libero.it>
> 
> The pdumper branch has not yet been merged to master.
> 
> Is there anything holding up the merge?

Not much:

  http://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00924.html
  http://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00929.html
  http://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00974.html

Re: Preview: portable dumper

[Alan Third]

On Fri, Jun 08, 2018 at 01:05:36PM +0300, Eli Zaretskii wrote:
> > From: Robert Pluim <rpluim@gmail.com>
> > Date: Fri, 08 Jun 2018 11:29:29 +0200
> > Cc: Alan Third <alan@idiocy.org>, Daniel Colascione <dancol@dancol.org>,
> > 	Angelo Graziosi <angelo.g0@libero.it>
> > 
> > The pdumper branch has not yet been merged to master.
> > 
> > Is there anything holding up the merge?
> 
> Not much:
> 
>   http://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00924.html
>   http://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00929.html
>   http://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00974.html

I pushed the patch in the third link to the pdumper branch a while
back since nobody commented.

-- 
Alan Third

Re: Preview: portable dumper

[Eli Zaretskii]

> Date: Tue, 13 Feb 2018 23:06:15 +0100 (CET)
> From: Angelo Graziosi <angelo.g0@libero.it>
> 
> Eli Zaretskii wrote:
> >
> > I'd urge people to try this branch and report any issues they see.
> 
> What should we look for?

The usual stuff, I think: does it build, does it work as expected,
etc.

Re: Preview: portable dumper

[Robert Pluim]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Tue, 13 Feb 2018 23:06:15 +0100 (CET)
>> From: Angelo Graziosi <angelo.g0@libero.it>
>> 
>> Eli Zaretskii wrote:
>> >
>> > I'd urge people to try this branch and report any issues they see.
>> 
>> What should we look for?
>
> The usual stuff, I think: does it build, does it work as expected,
> etc.

You mean like:

make[3]: Entering directory '/home/rpluim/repos/emacs-pdumper/admin/grammars'
  GEN      ../../lisp/cedet/semantic/bovine/c-by.el
load_dump completed in 63.0991 milliseconds
../../lisp/emacs-lisp/eieio.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
../../lisp/emacs-lisp/eieio-base.el: ‘eieio-object-set-name-string’ is an obsolete generic function (as of 25.1); inherit from ‘eieio-named’ and use (setf (slot-value OBJ ’object-name) NAME) instead
../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
../../lisp/cedet/semantic/db-ref.el: Obsolete name arg "DEBUG" to constructor semanticdb-ref-adebug
Fatal error 11: Segmentation fault

This is with gcc 5.4.0 on Ubuntu 16.04. I rebuilt with '-O0 -ggdb' and
trapped it in gdb.

(gdb) l
4534	  int bits_per_word = xword_size * CHAR_BIT;
4535	  dump_bitset_word * slot = dump_bitset__bit_slot (bitset, bit_number);
4536	  dump_bitset_word bit = 1;
4537	  bit <<= bit_number % bits_per_word;
4538	  if (bit_is_set)
4539	    *slot = *slot | bit;
4540	  else
4541	    *slot = *slot & ~bit;
4542	}

(gdb) p slot
$1 = (dump_bitset_word *) 0x7ffff7ffb3a8
(gdb) p *slot
$2 = 12
(gdb) p bit
$3 = 134217728
(gdb) p bits_per_word 
$4 = 64
(gdb) p bit_number 
$5 = 1711323


(gdb) backtrace 
#0  0x000000000060015e in dump_bitset__set_bit_value (bitset=0xce6690 <dump_private+80>, bit_number=1711323, 
    bit_is_set=true) at pdumper.c:4539
#1  0x00000000006001a6 in dump_bitset_set_bit (bitset=0xce6690 <dump_private+80>, bit_number=1711323)
    at pdumper.c:4547
#2  0x00000000006004e7 in pdumper_set_marked_impl (obj=0x7fffe539b6d8) at pdumper.c:4704
#3  0x0000000000568f28 in pdumper_set_marked (obj=0x7fffe539b6d8) at pdumper.h:222
#4  0x00000000005f1941 in set_symbol_marked (s=0x7fffe539b6d8) at alloc.c:4205
#5  0x00000000005f600a in mark_object (arg=XIL(0x7fffe46b41f8)) at alloc.c:6997
#6  0x00000000005f2d52 in mark_maybe_object (obj=XIL(0x7fffe46b41f8)) at alloc.c:5018
#7  0x00000000005f30ef in mark_memory (start=0x7ffffffdf580, end=0x7fffffffd870) at alloc.c:5210
#8  0x00000000005f3124 in mark_stack (bottom=0x7fffffffd870 "", end=0x7ffffffdf580 "\220\365\375\377\377\177")
    at alloc.c:5417
#9  0x00000000006aa9d7 in mark_one_thread (thread=0xd0b400 <main_thread>) at thread.c:617
#10 0x00000000006aaad8 in mark_threads_callback (ignore=0x0) at thread.c:650
#11 0x00000000005f317a in flush_stack_call_func (func=0x6aaa92 <mark_threads_callback>, arg=0x0) at alloc.c:5444
#12 0x00000000006aab04 in mark_threads () at thread.c:657
#13 0x00000000005f4dd4 in garbage_collect_1 (end=0x7ffffffdf780) at alloc.c:6322
#14 0x00000000005f54a6 in Fgarbage_collect () at alloc.c:6493
#15 0x0000000000566abd in maybe_gc () at lisp.h:5002
#16 0x0000000000623523 in eval_sub (form=XIL(0x7fffe47709e3)) at eval.c:2195
#17 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#18 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe47709a3), nargs=2, arg_vector=0x7ffffffdfa10) at eval.c:3098
#19 0x0000000000625389 in Ffuncall (nargs=3, args=0x7ffffffdfa08) at eval.c:2836
#20 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7ffffffdff50) at bytecode.c:632
#21 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7ffffffdff40) at eval.c:3023
#22 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffdff38) at eval.c:2824
#23 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffe0bc0) at bytecode.c:632
#24 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffe0bb8) at eval.c:3023
#25 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe0bb0) at eval.c:2824
#26 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffe10a8) at bytecode.c:632
#27 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffe1098) at eval.c:3023
#28 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe1090) at eval.c:2824
#29 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7ffffffe15d0) at bytecode.c:632
#30 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7ffffffe15c0) at eval.c:3023
#31 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe15b8) at eval.c:2824
#32 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffe2240) at bytecode.c:632
#33 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffe2238) at eval.c:3023
#34 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe2230) at eval.c:2824
#35 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffe2728) at bytecode.c:632
#36 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffe2718) at eval.c:3023
#37 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe2710) at eval.c:2824
#38 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7ffffffe2c50) at bytecode.c:632
#39 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7ffffffe2c40) at eval.c:3023
#40 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe2c38) at eval.c:2824
#41 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffe38c0) at bytecode.c:632
#42 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffe38b8) at eval.c:3023
#43 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe38b0) at eval.c:2824
#44 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffe3dc8) at bytecode.c:632
#45 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffe3db8) at eval.c:3023
#46 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe3db0) at eval.c:2824
#47 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4bbadd4), vector=XIL(0x7fffe4bbadbd), 
    maxdepth=make_number(10), args_template=make_number(513), nargs=1, args=0x7ffffffe42e8) at bytecode.c:632
#48 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4bbad7d), nargs=1, arg_vector=0x7ffffffe42e0) at eval.c:3023
#49 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe42d8) at eval.c:2824
#50 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffe4f40) at bytecode.c:632
#51 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffe4f38) at eval.c:3023
#52 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe4f30) at eval.c:2824
#53 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=1, args=0x7ffffffe5448) at bytecode.c:632
#54 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=1, arg_vector=0x7ffffffe5440) at eval.c:3023
#55 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe5438) at eval.c:2824
#56 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d4b4), vector=XIL(0x7fffe506d48d), 
    maxdepth=make_number(10), args_template=make_number(1028), nargs=4, args=0x7ffffffe5938) at bytecode.c:632
#57 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d43d), nargs=4, arg_vector=0x7ffffffe5918) at eval.c:3023
#58 0x00000000006252a9 in Ffuncall (nargs=5, args=0x7ffffffe5910) at eval.c:2824
#59 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffe6580) at bytecode.c:632
#60 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffe6578) at eval.c:3023
#61 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe6570) at eval.c:2824
#62 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffe6a68) at bytecode.c:632
#63 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffe6a58) at eval.c:3023
#64 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe6a50) at eval.c:2824
#65 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7ffffffe6f90) at bytecode.c:632
#66 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7ffffffe6f80) at eval.c:3023
#67 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe6f78) at eval.c:2824
#68 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffe7c00) at bytecode.c:632
#69 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffe7bf8) at eval.c:3023
#70 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe7bf0) at eval.c:2824
#71 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=1, args=0x7ffffffe8108) at bytecode.c:632
#72 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=1, arg_vector=0x7ffffffe8100) at eval.c:3023
#73 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe80f8) at eval.c:2824
#74 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d4b4), vector=XIL(0x7fffe506d48d), 
    maxdepth=make_number(10), args_template=make_number(1028), nargs=4, args=0x7ffffffe85f8) at bytecode.c:632
#75 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d43d), nargs=4, arg_vector=0x7ffffffe85d8) at eval.c:3023
#76 0x00000000006252a9 in Ffuncall (nargs=5, args=0x7ffffffe85d0) at eval.c:2824
#77 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffe9240) at bytecode.c:632
#78 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffe9238) at eval.c:3023
#79 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe9230) at eval.c:2824
#80 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffe9748) at bytecode.c:632
#81 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffe9738) at eval.c:3023
#82 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffe9730) at eval.c:2824
#83 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4bbadd4), vector=XIL(0x7fffe4bbadbd), 
    maxdepth=make_number(10), args_template=make_number(513), nargs=1, args=0x7ffffffe9c68) at bytecode.c:632
#84 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4bbad7d), nargs=1, arg_vector=0x7ffffffe9c60) at eval.c:3023
#85 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffe9c58) at eval.c:2824
#86 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffea8c0) at bytecode.c:632
#87 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffea8b8) at eval.c:3023
#88 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffea8b0) at eval.c:2824
#89 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=1, args=0x7ffffffeadc8) at bytecode.c:632
#90 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=1, arg_vector=0x7ffffffeadc0) at eval.c:3023
#91 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffeadb8) at eval.c:2824
#92 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d4b4), vector=XIL(0x7fffe506d48d), 
    maxdepth=make_number(10), args_template=make_number(1028), nargs=4, args=0x7ffffffeb2b8) at bytecode.c:632
#93 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d43d), nargs=4, arg_vector=0x7ffffffeb298) at eval.c:3023
#94 0x00000000006252a9 in Ffuncall (nargs=5, args=0x7ffffffeb290) at eval.c:2824
#95 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffebf00) at bytecode.c:632
#96 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffebef8) at eval.c:3023
#97 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffebef0) at eval.c:2824
#98 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffec3e8) at bytecode.c:632
#99 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffec3d8) at eval.c:3023
#100 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffec3d0) at eval.c:2824
#101 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7ffffffec910) at bytecode.c:632
#102 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7ffffffec900)
    at eval.c:3023
#103 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffec8f8) at eval.c:2824
#104 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffed580) at bytecode.c:632
#105 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffed578)
    at eval.c:3023
#106 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffed570) at eval.c:2824
#107 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffeda68) at bytecode.c:632
#108 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffeda58)
    at eval.c:3023
#109 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffeda50) at eval.c:2824
#110 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7ffffffedf90) at bytecode.c:632
#111 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7ffffffedf80)
    at eval.c:3023
#112 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffedf78) at eval.c:2824
#113 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7ffffffeec00) at bytecode.c:632
#114 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7ffffffeebf8)
    at eval.c:3023
#115 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7ffffffeebf0) at eval.c:2824
#116 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7ffffffef0e8) at bytecode.c:632
#117 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7ffffffef0d8)
    at eval.c:3023
#118 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffef0d0) at eval.c:2824
#119 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7ffffffef610) at bytecode.c:632
#120 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7ffffffef600)
    at eval.c:3023
#121 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7ffffffef5f8) at eval.c:2824
#122 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7fffffff0280) at bytecode.c:632
#123 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7fffffff0278)
    at eval.c:3023
#124 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7fffffff0270) at eval.c:2824
#125 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=1, args=0x7fffffff0788) at bytecode.c:632
#126 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=1, arg_vector=0x7fffffff0780)
    at eval.c:3023
#127 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7fffffff0778) at eval.c:2824
#128 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d4b4), vector=XIL(0x7fffe506d48d), 
    maxdepth=make_number(10), args_template=make_number(1028), nargs=4, args=0x7fffffff0c78) at bytecode.c:632
#129 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d43d), nargs=4, arg_vector=0x7fffffff0c58)
    at eval.c:3023
#130 0x00000000006252a9 in Ffuncall (nargs=5, args=0x7fffffff0c50) at eval.c:2824
#131 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7fffffff18c0) at bytecode.c:632
#132 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7fffffff18b8)
    at eval.c:3023
#133 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7fffffff18b0) at eval.c:2824
#134 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7fffffff1e10) at bytecode.c:632
#135 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7fffffff1e00)
    at eval.c:3023
#136 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7fffffff1df8) at eval.c:2824
#137 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7fffffff2a40) at bytecode.c:632
#138 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7fffffff2a38)
    at eval.c:3023
#139 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7fffffff2a30) at eval.c:2824
#140 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4a8b2b4), vector=XIL(0x7fffe4a8b295), 
    maxdepth=make_number(9), args_template=make_number(513), nargs=2, args=0x7fffffff2f28) at bytecode.c:632
#141 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4a8b255), nargs=2, arg_vector=0x7fffffff2f18)
    at eval.c:3023
#142 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7fffffff2f10) at eval.c:2824
#143 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d5cc), vector=XIL(0x7fffe506d525), 
    maxdepth=make_number(8), args_template=make_number(514), nargs=2, args=0x7fffffff3450) at bytecode.c:632
#144 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe506d4d5), nargs=2, arg_vector=0x7fffffff3440)
    at eval.c:3023
#145 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7fffffff3438) at eval.c:2824
#146 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d734), vector=XIL(0x7fffe470a44d), 
    maxdepth=make_number(28), args_template=make_number(257), nargs=1, args=0x7fffffff4098) at bytecode.c:632
#147 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a40d), nargs=1, arg_vector=0x7fffffff4090)
    at eval.c:3023
#148 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7fffffff4088) at eval.c:2824
#149 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe506d774), vector=XIL(0x7fffe470a375), 
    maxdepth=make_number(4), args_template=make_number(513), nargs=1, args=0x7fffffff4518) at bytecode.c:632
#150 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe470a335), nargs=1, arg_vector=0x7fffffff4510)
    at eval.c:3023
#151 0x00000000006252a9 in Ffuncall (nargs=2, args=0x7fffffff4508) at eval.c:2824
#152 0x000000000066fd76 in exec_byte_code (bytestr=XIL(0x7fffe4d23a34), vector=XIL(0x7fffe4d22d85), 
    maxdepth=make_number(11), args_template=make_number(514), nargs=2, args=0x7fffffff4a58) at bytecode.c:632
#153 0x0000000000625d1f in funcall_lambda (fun=XIL(0x7fffe4d22d45), nargs=2, arg_vector=0x7fffffff4a48)
    at eval.c:3023
#154 0x00000000006252a9 in Ffuncall (nargs=3, args=0x7fffffff4a40) at eval.c:2824
#155 0x0000000000624b6e in call2 (fn=XIL(0x7fffe403b830), arg1=XIL(0x1231393), arg2=XIL(0xc480)) at eval.c:2681
#156 0x000000000065548a in readevalloop_eager_expand_eval (val=XIL(0x1231393), macroexpand=XIL(0x7fffe403b830))
    at lread.c:1885
#157 0x0000000000655e24 in readevalloop (readcharfun=XIL(0x12b63b5), infile0=0x0, sourcename=XIL(0x12b7304), 
    printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2071
#158 0x000000000065624e in Feval_buffer (buffer=XIL(0x12b63b5), printflag=XIL(0), filename=XIL(0x12b5b94), 
    unibyte=XIL(0), do_allow_print=XIL(0xc480)) at lread.c:2138
#159 0x0000000000623c97 in eval_sub (form=XIL(0x7fffe4746a73)) at eval.c:2301
#160 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#161 0x0000000000620eba in Flet (args=XIL(0x7fffe4746a43)) at eval.c:973
#162 0x00000000006237be in eval_sub (form=XIL(0x7fffe4746a33)) at eval.c:2239
#163 0x00000000006213b2 in Funwind_protect (args=XIL(0x7fffe47469a3)) at eval.c:1195
#164 0x00000000006237be in eval_sub (form=XIL(0x7fffe4746993)) at eval.c:2239
#165 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe4746703)) at eval.c:459
#166 0x0000000000620abf in FletX (args=XIL(0x7fffe47466c3)) at eval.c:904
#167 0x00000000006237be in eval_sub (form=XIL(0x7fffe47466b3)) at eval.c:2239
#168 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#169 0x000000000061fb41 in Fif (args=XIL(0x7fffe4746683)) at eval.c:415
#170 0x00000000006237be in eval_sub (form=XIL(0x7fffe4746673)) at eval.c:2239
#171 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#172 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4746633), nargs=4, arg_vector=0x7fffffff5508)
    at eval.c:3098
#173 0x0000000000625389 in Ffuncall (nargs=5, args=0x7fffffff5500) at eval.c:2836
#174 0x0000000000624c56 in call4 (fn=XIL(0x7fffe3a5f120), arg1=XIL(0x12b5b94), arg2=XIL(0x12b5b94), arg3=XIL(0), 
    arg4=XIL(0xc480)) at eval.c:2698
#175 0x0000000000654040 in Fload (file=XIL(0x7fffe480b71c), noerror=XIL(0), nomessage=XIL(0xc480), nosuffix=XIL(0), 
    must_suffix=XIL(0xc480)) at lread.c:1367
#176 0x0000000000631ed7 in Frequire (feature=XIL(0x7fffe3b24178), filename=XIL(0), noerror=XIL(0)) at fns.c:2807
#177 0x0000000000623c31 in eval_sub (form=XIL(0x122e753)) at eval.c:2293
#178 0x0000000000655492 in readevalloop_eager_expand_eval (val=XIL(0x122e753), macroexpand=XIL(0x7fffe403b830))
    at lread.c:1885
#179 0x0000000000655e24 in readevalloop (readcharfun=XIL(0x115d015), infile0=0x0, sourcename=XIL(0x1116c34), 
    printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2071
#180 0x000000000065624e in Feval_buffer (buffer=XIL(0x115d015), printflag=XIL(0), filename=XIL(0x112fd64), 
    unibyte=XIL(0), do_allow_print=XIL(0xc480)) at lread.c:2138
#181 0x0000000000623c97 in eval_sub (form=XIL(0x7fffe4746a73)) at eval.c:2301
#182 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#183 0x0000000000620eba in Flet (args=XIL(0x7fffe4746a43)) at eval.c:973
#184 0x00000000006237be in eval_sub (form=XIL(0x7fffe4746a33)) at eval.c:2239
#185 0x00000000006213b2 in Funwind_protect (args=XIL(0x7fffe47469a3)) at eval.c:1195
#186 0x00000000006237be in eval_sub (form=XIL(0x7fffe4746993)) at eval.c:2239
#187 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe4746703)) at eval.c:459
#188 0x0000000000620abf in FletX (args=XIL(0x7fffe47466c3)) at eval.c:904
#189 0x00000000006237be in eval_sub (form=XIL(0x7fffe47466b3)) at eval.c:2239
#190 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#191 0x000000000061fb41 in Fif (args=XIL(0x7fffe4746683)) at eval.c:415
#192 0x00000000006237be in eval_sub (form=XIL(0x7fffe4746673)) at eval.c:2239
#193 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#194 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4746633), nargs=4, arg_vector=0x7fffffff6398)
    at eval.c:3098
#195 0x0000000000625389 in Ffuncall (nargs=5, args=0x7fffffff6390) at eval.c:2836
#196 0x0000000000624c56 in call4 (fn=XIL(0x7fffe3a5f120), arg1=XIL(0x112fd64), arg2=XIL(0x112fd64), arg3=XIL(0), 
    arg4=XIL(0xc480)) at eval.c:2698
#197 0x0000000000654040 in Fload (file=XIL(0x7fffe48e1324), noerror=XIL(0), nomessage=XIL(0xc480), nosuffix=XIL(0), 
    must_suffix=XIL(0)) at lread.c:1367
#198 0x0000000000623c97 in eval_sub (form=XIL(0x7fffe48e12e3)) at eval.c:2301
#199 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe48e12b3)) at eval.c:459
#200 0x00000000006237be in eval_sub (form=XIL(0x7fffe48e1293)) at eval.c:2239
#201 0x000000000061fb23 in Fif (args=XIL(0x7fffe48e1273)) at eval.c:414
#202 0x00000000006237be in eval_sub (form=XIL(0x7fffe48e1263)) at eval.c:2239
#203 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#204 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe48e1223), nargs=1, arg_vector=0x7fffffff6b68)
    at eval.c:3098
#205 0x0000000000625389 in Ffuncall (nargs=2, args=0x7fffffff6b60) at eval.c:2836
#206 0x0000000000624173 in Fapply (nargs=2, args=0x7fffffff6b60) at eval.c:2399
#207 0x00000000006239f5 in eval_sub (form=XIL(0x7fffe48817b3)) at eval.c:2258
#208 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#209 0x000000000061fbcc in Fcond (args=XIL(0x7fffe4881783)) at eval.c:439
#210 0x00000000006237be in eval_sub (form=XIL(0x7fffe4881753)) at eval.c:2239
#211 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#212 0x0000000000620eba in Flet (args=XIL(0x7fffe4881723)) at eval.c:973
#213 0x00000000006237be in eval_sub (form=XIL(0x7fffe4881713)) at eval.c:2239
#214 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#215 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe48816d3), nargs=3, arg_vector=0x7fffffff7110)
    at eval.c:3098
#216 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe48816d3), args=XIL(0x7fffe4e776bb), count=89) at eval.c:2959
#217 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe4e776ab)) at eval.c:2362
#218 0x000000000061fa9f in Fand (args=XIL(0x7fffe4e775db)) at eval.c:393
#219 0x00000000006237be in eval_sub (form=XIL(0x7fffe4e775bb)) at eval.c:2239
#220 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#221 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4e7758b), nargs=1, arg_vector=0x7fffffff7578)
    at eval.c:3098
#222 0x0000000000625389 in Ffuncall (nargs=2, args=0x7fffffff7570) at eval.c:2836
#223 0x0000000000624b0c in call1 (fn=XIL(0x7fffe4e7758b), arg1=XIL(0x7fffe3c13f68)) at eval.c:2673
#224 0x0000000000630e6c in mapcar1 (leni=9, vals=0x0, fn=XIL(0x7fffe4e7758b), seq=XIL(0x7fffe48faebb)) at fns.c:2486
#225 0x0000000000631359 in Fmapc (function=XIL(0x7fffe4e7758b), sequence=XIL(0x7fffe48faebb)) at fns.c:2555
#226 0x0000000000623c01 in eval_sub (form=XIL(0x7fffe4e7742b)) at eval.c:2290
#227 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#228 0x0000000000620eba in Flet (args=XIL(0x7fffe4e7740b)) at eval.c:973
#229 0x00000000006237be in eval_sub (form=XIL(0x7fffe4e773fb)) at eval.c:2239
#230 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe4e7735b)) at eval.c:459
#231 0x000000000062117e in internal_catch (tag=XIL(0x7fffe3a39ed8), func=0x61fc00 <Fprogn>, arg=XIL(0x7fffe4e7734b))
    at eval.c:1101
#232 0x0000000000621132 in Fcatch (args=XIL(0x7fffe4e7733b)) at eval.c:1078
#233 0x00000000006237be in eval_sub (form=XIL(0x7fffe4e7732b)) at eval.c:2239
#234 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#235 0x000000000061fbcc in Fcond (args=XIL(0x7fffe4e772fb)) at eval.c:439
#236 0x00000000006237be in eval_sub (form=XIL(0x7fffe4e772cb)) at eval.c:2239
#237 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#238 0x0000000000620eba in Flet (args=XIL(0x7fffe4e772ab)) at eval.c:973
#239 0x00000000006237be in eval_sub (form=XIL(0x7fffe4e7729b)) at eval.c:2239
#240 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#241 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4e7725b), nargs=1, arg_vector=0x7fffffff7fc0)
    at eval.c:3098
#242 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe4e7725b), args=XIL(0x7fffe4fe5c53), count=77) at eval.c:2959
#243 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe4fe5c43)) at eval.c:2362
#244 0x000000000061faed in Fif (args=XIL(0x7fffe4fe5bc3)) at eval.c:411
#245 0x00000000006237be in eval_sub (form=XIL(0x7fffe4fe5bb3)) at eval.c:2239
#246 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#247 0x000000000061fbcc in Fcond (args=XIL(0x7fffe4fe5b83)) at eval.c:439
#248 0x00000000006237be in eval_sub (form=XIL(0x7fffe4fe5b53)) at eval.c:2239
#249 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#250 0x0000000000620eba in Flet (args=XIL(0x7fffe4fe5b33)) at eval.c:973
#251 0x00000000006237be in eval_sub (form=XIL(0x7fffe4fe5b23)) at eval.c:2239
#252 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#253 0x000000000061fbcc in Fcond (args=XIL(0x7fffe4882ec3)) at eval.c:439
#254 0x00000000006237be in eval_sub (form=XIL(0x7fffe4882eb3)) at eval.c:2239
#255 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#256 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4882e73), nargs=1, arg_vector=0x7fffffff88e0)
    at eval.c:3098
#257 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe4882e73), args=XIL(0x7fffe5039a0b), count=70) at eval.c:2959
#258 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe50399fb)) at eval.c:2362
#259 0x000000000062183c in internal_lisp_condition_case (var=XIL(0x7fffe43524c8), bodyform=XIL(0x7fffe50399fb), 
    handlers=XIL(0x7fffe503993b)) at eval.c:1307
#260 0x000000000062142c in Fcondition_case (args=XIL(0x7fffe503991b)) at eval.c:1231
#261 0x00000000006237be in eval_sub (form=XIL(0x7fffe503990b)) at eval.c:2239
#262 0x000000000061fda4 in Fsetq (args=XIL(0x7fffe50398eb)) at eval.c:517
#263 0x00000000006237be in eval_sub (form=XIL(0x7fffe50398db)) at eval.c:2239
#264 0x000000000061fb80 in Fcond (args=XIL(0x7fffe477493b)) at eval.c:435
#265 0x00000000006237be in eval_sub (form=XIL(0x7fffe477492b)) at eval.c:2239
#266 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#267 0x0000000000620eba in Flet (args=XIL(0x7fffe477490b)) at eval.c:973
#268 0x00000000006237be in eval_sub (form=XIL(0x7fffe47748fb)) at eval.c:2239
#269 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#270 0x00000000006237be in eval_sub (form=XIL(0x7fffe47748bb)) at eval.c:2239
#271 0x000000000061fb23 in Fif (args=XIL(0x7fffe477489b)) at eval.c:414
#272 0x00000000006237be in eval_sub (form=XIL(0x7fffe477488b)) at eval.c:2239
#273 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#274 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe477482b), nargs=0, arg_vector=0x7fffffff9670)
    at eval.c:3098
#275 0x0000000000625389 in Ffuncall (nargs=1, args=0x7fffffff9668) at eval.c:2836
#276 0x0000000000624541 in funcall_nil (nargs=1, args=0x7fffffff9668) at eval.c:2453
#277 0x0000000000624963 in run_hook_with_args (nargs=1, args=0x7fffffff9668, funcall=0x62451e <funcall_nil>)
    at eval.c:2630
#278 0x00000000006245c8 in Frun_hook_with_args (nargs=1, args=0x7fffffff9668) at eval.c:2495
#279 0x00000000006249c8 in run_hook (hook=XIL(0x7fffe3a8d318)) at eval.c:2643
#280 0x0000000000624585 in Frun_hooks (nargs=1, args=0x7fffffff96b0) at eval.c:2477
#281 0x00000000006239f5 in eval_sub (form=XIL(0x7fffe48519cb)) at eval.c:2258
#282 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#283 0x000000000061fb41 in Fif (args=XIL(0x10f1803)) at eval.c:415
#284 0x00000000006237be in eval_sub (form=XIL(0x10f13d3)) at eval.c:2239
#285 0x0000000000623f7a in eval_sub (form=XIL(0x7fffe485195b)) at eval.c:2358
#286 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#287 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe48518db), nargs=2, arg_vector=0x7fffffff9b50)
    at eval.c:3098
#288 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe48518cb), args=XIL(0x7fffe485185b), count=56) at eval.c:2959
#289 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe485184b)) at eval.c:2362
#290 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#291 0x000000000061fb41 in Fif (args=XIL(0x7fffe485181b)) at eval.c:415
#292 0x00000000006237be in eval_sub (form=XIL(0x7fffe485180b)) at eval.c:2239
#293 0x000000000061fc47 in Fprogn (body=XIL(0x104ed43)) at eval.c:459
#294 0x000000000060ee42 in Fsave_current_buffer (args=XIL(0x10700a3)) at editfns.c:1065
#295 0x00000000006237be in eval_sub (form=XIL(0x1070093)) at eval.c:2239
#296 0x0000000000623f7a in eval_sub (form=XIL(0x7fffe485170b)) at eval.c:2358
#297 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#298 0x0000000000620eba in Flet (args=XIL(0x7fffe48516eb)) at eval.c:973
#299 0x00000000006237be in eval_sub (form=XIL(0x7fffe48516db)) at eval.c:2239
#300 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#301 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe48516ab), nargs=6, arg_vector=0x7fffffffa420)
    at eval.c:3098
#302 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe485169b), args=XIL(0x7fffe485156b), count=48) at eval.c:2959
#303 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe485155b)) at eval.c:2362
#304 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#305 0x000000000061fb41 in Fif (args=XIL(0x7fffe485151b)) at eval.c:415
#306 0x00000000006237be in eval_sub (form=XIL(0x7fffe485150b)) at eval.c:2239
#307 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#308 0x0000000000620abf in FletX (args=XIL(0x7fffe48514cb)) at eval.c:904
#309 0x00000000006237be in eval_sub (form=XIL(0x7fffe48514bb)) at eval.c:2239
#310 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#311 0x000000000061fb41 in Fif (args=XIL(0x7fffe485148b)) at eval.c:415
#312 0x00000000006237be in eval_sub (form=XIL(0x7fffe485147b)) at eval.c:2239
#313 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#314 0x000000000061fb41 in Fif (args=XIL(0x7fffe485144b)) at eval.c:415
#315 0x00000000006237be in eval_sub (form=XIL(0x7fffe485143b)) at eval.c:2239
#316 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#317 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe48513eb), nargs=1, arg_vector=0x7fffffffad10)
    at eval.c:3098
#318 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe48513db), args=XIL(0x10650e3), count=41) at eval.c:2959
#319 0x0000000000623fd5 in eval_sub (form=XIL(0x1065403)) at eval.c:2362
#320 0x0000000000623afa in eval_sub (form=XIL(0x1069183)) at eval.c:2275
#321 0x000000000061fc47 in Fprogn (body=XIL(0x1069b23)) at eval.c:459
#322 0x000000000060ee42 in Fsave_current_buffer (args=XIL(0x1069173)) at editfns.c:1065
#323 0x00000000006237be in eval_sub (form=XIL(0x1069163)) at eval.c:2239
#324 0x000000000062183c in internal_lisp_condition_case (var=XIL(0x7fffe39d9288), bodyform=XIL(0x1069163), 
    handlers=XIL(0x1064683)) at eval.c:1307
#325 0x000000000062142c in Fcondition_case (args=XIL(0x1068a83)) at eval.c:1231
#326 0x00000000006237be in eval_sub (form=XIL(0x1068a73)) at eval.c:2239
#327 0x0000000000620d08 in Flet (args=XIL(0x1058b13)) at eval.c:943
#328 0x00000000006237be in eval_sub (form=XIL(0x1058b03)) at eval.c:2239
#329 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#330 0x0000000000626016 in funcall_lambda (fun=XIL(0x1058af3), nargs=2, arg_vector=0x7fffffffb690) at eval.c:3098
#331 0x0000000000625a79 in apply_lambda (fun=XIL(0x1058af3), args=XIL(0x1065b33), count=33) at eval.c:2959
#332 0x0000000000623fd5 in eval_sub (form=XIL(0x1065b43)) at eval.c:2362
#333 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#334 0x0000000000626016 in funcall_lambda (fun=XIL(0x104a2c3), nargs=0, arg_vector=0x7fffffffb988) at eval.c:3098
#335 0x0000000000625389 in Ffuncall (nargs=1, args=0x7fffffffb980) at eval.c:2836
#336 0x00000000006239f5 in eval_sub (form=XIL(0x7fffe4be0e9b)) at eval.c:2258
#337 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#338 0x000000000061fb41 in Fif (args=XIL(0x7fffe4be0e6b)) at eval.c:415
#339 0x00000000006237be in eval_sub (form=XIL(0x7fffe4be0e5b)) at eval.c:2239
#340 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#341 0x000000000061fbcc in Fcond (args=XIL(0x7fffe4bde66b)) at eval.c:439
#342 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bde63b)) at eval.c:2239
#343 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe4bde59b)) at eval.c:459
#344 0x0000000000620abf in FletX (args=XIL(0x7fffe4bde55b)) at eval.c:904
#345 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bde54b)) at eval.c:2239
#346 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#347 0x000000000061fc75 in prog_ignore (body=XIL(0x7fffe4bde53b)) at eval.c:470
#348 0x0000000000620f56 in Fwhile (args=XIL(0x7fffe4bde52b)) at eval.c:992
#349 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bde51b)) at eval.c:2239
#350 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#351 0x0000000000620abf in FletX (args=XIL(0x7fffe4bde4db)) at eval.c:904
#352 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bde4cb)) at eval.c:2239
#353 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#354 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bde4ab)) at eval.c:2239
#355 0x000000000061fb23 in Fif (args=XIL(0x7fffe4bde48b)) at eval.c:414
#356 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bde47b)) at eval.c:2239
#357 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#358 0x0000000000620eba in Flet (args=XIL(0x7fffe4bde45b)) at eval.c:973
#359 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bde44b)) at eval.c:2239
#360 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe4bc59f3)) at eval.c:459
#361 0x0000000000620eba in Flet (args=XIL(0x7fffe4bc59d3)) at eval.c:973
#362 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bc59c3)) at eval.c:2239
#363 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#364 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4bc5963), nargs=1, arg_vector=0x7fffffffcaa0)
    at eval.c:3098
#365 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe4bc5953), args=XIL(0x7fffe4bc58f3), count=13) at eval.c:2959
#366 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe4bc58e3)) at eval.c:2362
#367 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe4bc4223)) at eval.c:459
#368 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4bc4083), nargs=0, arg_vector=0x7fffffffcd30)
    at eval.c:3098
#369 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe4bc4073), args=XIL(0), count=11) at eval.c:2959
#370 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe4bc4033)) at eval.c:2362
#371 0x00000000006213b2 in Funwind_protect (args=XIL(0x7fffe4bc1073)) at eval.c:1195
#372 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bc1063)) at eval.c:2239
#373 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#374 0x0000000000620eba in Flet (args=XIL(0x7fffe4bc1043)) at eval.c:973
#375 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bc1033)) at eval.c:2239
#376 0x000000000061fc47 in Fprogn (body=XIL(0x7fffe4bc0a23)) at eval.c:459
#377 0x000000000061fb41 in Fif (args=XIL(0x7fffe4bc0953)) at eval.c:415
#378 0x00000000006237be in eval_sub (form=XIL(0x7fffe4bc0943)) at eval.c:2239
#379 0x000000000061fc47 in Fprogn (body=XIL(0)) at eval.c:459
#380 0x0000000000626016 in funcall_lambda (fun=XIL(0x7fffe4bc0903), nargs=0, arg_vector=0x7fffffffd4a0)
    at eval.c:3098
#381 0x0000000000625a79 in apply_lambda (fun=XIL(0x7fffe4bc08f3), args=XIL(0), count=4) at eval.c:2959
#382 0x0000000000623fd5 in eval_sub (form=XIL(0x7fffe526d9c3)) at eval.c:2362
#383 0x00000000006231f9 in Feval (form=XIL(0x7fffe526d9c3), lexical=XIL(0)) at eval.c:2107
#384 0x000000000056de9f in top_level_2 () at keyboard.c:1122
#385 0x00000000006218ec in internal_condition_case (bfun=0x56de7c <top_level_2>, handlers=XIL(0x53d0), 
    hfun=0x56d90d <cmd_error>) at eval.c:1336
#386 0x000000000056dee3 in top_level_1 (ignore=XIL(0)) at keyboard.c:1130
#387 0x000000000062117e in internal_catch (tag=XIL(0xcae0), func=0x56dea1 <top_level_1>, arg=XIL(0)) at eval.c:1101
#388 0x000000000056ddce in command_loop () at keyboard.c:1091
#389 0x000000000056d4e2 in recursive_edit_1 () at keyboard.c:698
#390 0x000000000056d664 in Frecursive_edit () at keyboard.c:769
#391 0x000000000056b500 in main (argc=11, argv=0x7fffffffda28) at emacs.c:1907

Lisp Backtrace:
"Automatic GC" (0x0)
"function-get" (0xfffdfa10)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xfffe0bb8)
"macroexp--all-forms" (0xfffe1098)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xfffe2238)
"macroexp--all-forms" (0xfffe2718)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xfffe38b8)
"macroexp--all-forms" (0xfffe3db8)
"macroexp--all-clauses" (0xfffe42e0)
"macroexp--expand-all" (0xfffe4f38)
"macroexp--all-forms" (0xfffe5440)
0xe506d438 PVEC_COMPILED
"macroexp--expand-all" (0xfffe6578)
"macroexp--all-forms" (0xfffe6a58)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xfffe7bf8)
"macroexp--all-forms" (0xfffe8100)
0xe506d438 PVEC_COMPILED
"macroexp--expand-all" (0xfffe9238)
"macroexp--all-forms" (0xfffe9738)
"macroexp--all-clauses" (0xfffe9c60)
"macroexp--expand-all" (0xfffea8b8)
"macroexp--all-forms" (0xfffeadc0)
0xe506d438 PVEC_COMPILED
"macroexp--expand-all" (0xfffebef8)
"macroexp--all-forms" (0xfffec3d8)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xfffed578)
"macroexp--all-forms" (0xfffeda58)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xfffeebf8)
"macroexp--all-forms" (0xfffef0d8)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xffff0278)
"macroexp--all-forms" (0xffff0780)
0xe506d438 PVEC_COMPILED
"macroexp--expand-all" (0xffff18b8)
"macroexp--all-forms" (0xffff1e00)
"macroexp--expand-all" (0xffff2a38)
"macroexp--all-forms" (0xffff2f18)
0xe506d4d0 PVEC_COMPILED
"macroexp--expand-all" (0xffff4090)
"macroexpand-all" (0xffff4510)
"internal-macroexpand-for-load" (0xffff4a48)
"eval-buffer" (0xffff4c80)
"let" (0xffff4e20)
"unwind-protect" (0xffff4f70)
"let*" (0xffff5130)
"if" (0xffff52b0)
"load-with-code-conversion" (0xffff5508)
"require" (0xffff5890)
"eval-buffer" (0xffff5b10)
"let" (0xffff5cb0)
"unwind-protect" (0xffff5e00)
"let*" (0xffff5fc0)
"if" (0xffff6140)
"load-with-code-conversion" (0xffff6398)
"load" (0xffff66a0)
"progn" (0xffff6760)
"if" (0xffff68b0)
"vc-git-registered" (0xffff6b68)
"apply" (0xffff6b60)
"cond" (0xffff6d30)
"let" (0xffff6f50)
"vc-call-backend" (0xffff7110)
"and" (0xffff7340)
0xe4e77588 Lisp type 3
"mapc" (0xffff76f0)
"let" (0xffff7880)
"catch" (0xffff7a50)
"cond" (0xffff7be0)
"let" (0xffff7e00)
"vc-registered" (0xffff7fc0)
"if" (0xffff81e0)
"cond" (0xffff8370)
"let" (0xffff8590)
"cond" (0xffff8720)
"vc-backend" (0xffff88e0)
"condition-case" (0xffff8c00)
"setq" (0xffff8d80)
"cond" (0xffff8ee0)
"let" (0xffff9100)
"progn" (0xffff9250)
"if" (0xffff93a0)
"vc-refresh-state" (0xffff9670)
"run-hooks" (0xffff96b0)
"if" (0xffff9870)
"unless" (0xffff9990)
"after-find-file" (0xffff9b50)
"if" (0xffff9da0)
"save-current-buffer" (0xffff9f20)
"with-current-buffer" (0xffffa040)
"let" (0xffffa260)
"find-file-noselect-1" (0xffffa420)
"if" (0xffffa690)
"let*" (0xffffa850)
"if" (0xffffa9d0)
"if" (0xffffab50)
"find-file-noselect" (0xffffad10)
"set-buffer" (0xffffaf00)
"save-current-buffer" (0xffffb080)
"condition-case" (0xffffb2d0)
"let" (0xffffb4d0)
"bovine--make-parser-1" (0xffffb690)
"bovine-batch-make-parser" (0xffffb988)
"funcall" (0xffffb980)
"if" (0xffffbb40)
"cond" (0xffffbcd0)
"let*" (0xffffbe90)
"while" (0xffffc040)
"let*" (0xffffc200)
"progn" (0xffffc350)
"if" (0xffffc4a0)
"let" (0xffffc6c0)
"let" (0xffffc8e0)
"command-line-1" (0xffffcaa0)
"command-line" (0xffffcd30)
"unwind-protect" (0xffffcf40)
"let" (0xffffd160)
"if" (0xffffd2e0)
"normal-top-level" (0xffffd4a0)

Re: Preview: portable dumper

[Daniel Colascione]

On 02/14/2018 02:30 AM, Robert Pluim wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
>>> Date: Tue, 13 Feb 2018 23:06:15 +0100 (CET)
>>> From: Angelo Graziosi <angelo.g0@libero.it>
>>>
>>> Eli Zaretskii wrote:
>>>>
>>>> I'd urge people to try this branch and report any issues they see.
>>>
>>> What should we look for?
>>
>> The usual stuff, I think: does it build, does it work as expected,
>> etc.
> 
> You mean like:
> 
> make[3]: Entering directory '/home/rpluim/repos/emacs-pdumper/admin/grammars'
>    GEN      ../../lisp/cedet/semantic/bovine/c-by.el
> load_dump completed in 63.0991 milliseconds
> ../../lisp/emacs-lisp/eieio.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
> ../../lisp/emacs-lisp/eieio-base.el: ‘eieio-object-set-name-string’ is an obsolete generic function (as of 25.1); inherit from ‘eieio-named’ and use (setf (slot-value OBJ ’object-name) NAME) instead
> ../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
> ../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
> ../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
> ../../lisp/cedet/semantic/db-ref.el: Obsolete name arg "DEBUG" to constructor semanticdb-ref-adebug
> Fatal error 11: Segmentation fault

Thanks. Did you try it with --enable-checking?

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

> On 02/14/2018 02:30 AM, Robert Pluim wrote:
>>
>> make[3]: Entering directory '/home/rpluim/repos/emacs-pdumper/admin/grammars'
>>    GEN      ../../lisp/cedet/semantic/bovine/c-by.el
>> load_dump completed in 63.0991 milliseconds
>> ../../lisp/emacs-lisp/eieio.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
>> ../../lisp/emacs-lisp/eieio-base.el: ‘eieio-object-set-name-string’
>> is an obsolete generic function (as of 25.1); inherit from
>> ‘eieio-named’ and use (setf (slot-value OBJ ’object-name) NAME)
>> instead
>> ../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
>> ../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
>> ../../lisp/cedet/semantic/db.el: ‘object-print’ is an obsolete generic function (as of 26.1); use ‘cl-print-object’ instead.
>> ../../lisp/cedet/semantic/db-ref.el: Obsolete name arg "DEBUG" to constructor semanticdb-ref-adebug
>> Fatal error 11: Segmentation fault
>
> Thanks. Did you try it with --enable-checking?

That gives me a different error:

Converting /home/rpluim/repos/emacs-pdumper/leim/CXTERM-DIC/ZOZY.tit to quail-package...
Decoding with coding system cn-big5...
Processing header part...
Formatting translation rules...
  GEN      ../lisp/leim/quail/tsang-b5.el
load_dump completed in 39.7511 milliseconds
Converting cangjie-table.b5 to tsang-b5.el...

pdumper.c:4691: Emacs fatal error: assertion failed: offset < dump_private.header.discardable_start
Fatal error 6: Aborted

Robert

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Wed, 14 Feb 2018 11:30:07 +0100
> 
> >> What should we look for?
> >
> > The usual stuff, I think: does it build, does it work as expected,
> > etc.
> 
> You mean like:

Yes, like that.

I see something slightly different on MS-Windows (in a build with
"--enable-checking"), but maybe similar enough to be explained by the
same problem:

       ELC      ../lisp/composite.elc
     load_dump completed in 46.005 milliseconds

     insdel.c:1937: Emacs fatal error: assertion failed: !pdumper_object_p (BEG_ADDR)

Backtrace:

  #0  0x762c3227 in KERNELBASE!DebugBreak ()
     from C:\Windows\syswow64\KernelBase.dll
  #1  0x0131940d in emacs_abort () at w32fns.c:10874
  #2  0x01152dea in terminate_due_to_signal (sig=22, backtrace_limit=2147483647)
      at emacs.c:388
  #3  0x01206274 in die (
      msg=0x16ce7d2 <DEFAULT_REHASH_SIZE+354> "!pdumper_object_p (BEG_ADDR)",
      file=0x16ce674 <DEFAULT_REHASH_SIZE+4> "insdel.c", line=1937)
      at alloc.c:7789
  #4  0x011acb8b in prepare_to_modify_buffer_1 (start=1, end=1,
      preserve_ptr=0x0) at insdel.c:1937
  #5  0x011acfa0 in prepare_to_modify_buffer (start=1, end=1, preserve_ptr=0x0)
      at insdel.c:1995
  #6  0x011c4b26 in Finsert_file_contents (filename=XIL(0x8000000008f797d0),
      visit=XIL(0), beg=XIL(0), end=XIL(0), replace=XIL(0)) at fileio.c:4142
  #7  0x0124343e in eval_sub (form=XIL(0xc000000007e47628)) at eval.c:2301
  #8  0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #9  0x0123e1cd in Flet (args=XIL(0xc000000007e47608)) at eval.c:973
  #10 0x01242bf4 in eval_sub (form=XIL(0xc000000007e475f8)) at eval.c:2239
  #11 0x0123b034 in Fprogn (body=XIL(0xc000000008f57860)) at eval.c:459
  #12 0x01226b99 in Fsave_current_buffer (args=XIL(0xc000000008f57010))
      at editfns.c:1065
  #13 0x01242bf4 in eval_sub (form=XIL(0xc000000008f57020)) at eval.c:2239
  #14 0x01243ad7 in eval_sub (form=XIL(0xc000000007e47468)) at eval.c:2358
  #15 0x0123b034 in Fprogn (body=XIL(0xc000000007e47368)) at eval.c:459
  #16 0x0123e1cd in Flet (args=XIL(0xc000000007e47348)) at eval.c:973
  #17 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47338)) at eval.c:2239
  #18 0x0123eca4 in Funwind_protect (args=XIL(0xc000000007e472a8))
      at eval.c:1195
  #19 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47298)) at eval.c:2239
  #20 0x0123b034 in Fprogn (body=XIL(0xc000000007e47090)) at eval.c:459
  #21 0x0123d8e6 in FletX (args=XIL(0xc000000007e47050)) at eval.c:904
  #22 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47040)) at eval.c:2239
  #23 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #24 0x0123accd in Fif (args=XIL(0xc000000007e47010)) at eval.c:415
  #25 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47000)) at eval.c:2239
  #26 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #27 0x01246f3a in funcall_lambda (fun=XIL(0xc000000007e46fc0), nargs=4,
      arg_vector=0x886be0) at eval.c:3098
  #28 0x01245840 in Ffuncall (nargs=5, args=0x886bd8) at eval.c:2836
  #29 0x01244ca9 in call4 (fn=XIL(0x66b30c8), arg1=XIL(0x8000000008f797d0),
      arg2=XIL(0x8000000008f797d0), arg3=XIL(0), arg4=XIL(0xed20))
      at eval.c:2698
  #30 0x01289c64 in Fload (file=XIL(0x8000000007be85b0), noerror=XIL(0),
      nomessage=XIL(0xed20), nosuffix=XIL(0), must_suffix=XIL(0xed20))
      at lread.c:1367
  #31 0x01258f3b in Frequire (feature=XIL(0x64546b8), filename=XIL(0),
      noerror=XIL(0)) at fns.c:2807
  #32 0x012432e8 in eval_sub (form=XIL(0xc000000008f4ef30)) at eval.c:2293
  #33 0x0128bd2f in readevalloop_eager_expand_eval (
      val=XIL(0xc000000008f4ef30), macroexpand=XIL(0x6a197d8)) at lread.c:1885
  #34 0x0128cbdc in readevalloop (readcharfun=XIL(0xa000000008f3c528),
      infile0=0x0, sourcename=XIL(0x8000000008f78e70), printflag=false,
      unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2071
  #35 0x0128d21d in Feval_buffer (buffer=XIL(0xa000000008f3c528),
      printflag=XIL(0), filename=XIL(0x8000000008f6d470), unibyte=XIL(0),
      do_allow_print=XIL(0xed20)) at lread.c:2138
  #36 0x0124343e in eval_sub (form=XIL(0xc000000007e47378)) at eval.c:2301
  #37 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #38 0x0123e1cd in Flet (args=XIL(0xc000000007e47348)) at eval.c:973
  #39 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47338)) at eval.c:2239
  #40 0x0123eca4 in Funwind_protect (args=XIL(0xc000000007e472a8))
      at eval.c:1195
  #41 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47298)) at eval.c:2239
  #42 0x0123b034 in Fprogn (body=XIL(0xc000000007e47090)) at eval.c:459
  #43 0x0123d8e6 in FletX (args=XIL(0xc000000007e47050)) at eval.c:904
  #44 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47040)) at eval.c:2239
  #45 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #46 0x0123accd in Fif (args=XIL(0xc000000007e47010)) at eval.c:415
  #47 0x01242bf4 in eval_sub (form=XIL(0xc000000007e47000)) at eval.c:2239
  #48 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #49 0x01246f3a in funcall_lambda (fun=XIL(0xc000000007e46fc0), nargs=4,
      arg_vector=0x888400) at eval.c:3098
  #50 0x01245840 in Ffuncall (nargs=5, args=0x8883f8) at eval.c:2836
  #51 0x01244ca9 in call4 (fn=XIL(0x66b30c8), arg1=XIL(0x8000000008f6d470),
      arg2=XIL(0x8000000008f6d470), arg3=XIL(0), arg4=XIL(0xed20))
      at eval.c:2698
  #52 0x01289c64 in Fload (file=XIL(0x8000000007bd9fa0), noerror=XIL(0),
      nomessage=XIL(0xed20), nosuffix=XIL(0), must_suffix=XIL(0xed20))
      at lread.c:1367
  #53 0x01258f3b in Frequire (feature=XIL(0x64460a8), filename=XIL(0),
      noerror=XIL(0)) at fns.c:2807
  #54 0x012432e8 in eval_sub (form=XIL(0xc000000007d997b0)) at eval.c:2293
  #55 0x0123b034 in Fprogn (body=XIL(0xc000000007d91060)) at eval.c:459
  #56 0x01246f3a in funcall_lambda (fun=XIL(0xc000000007d91020), nargs=1,
      arg_vector=0x888e10) at eval.c:3098
  #57 0x012463d3 in apply_lambda (fun=XIL(0xc000000007d91010),
      args=XIL(0xc0000000084a0fa0), count=72) at eval.c:2959
  #58 0x01243b7b in eval_sub (form=XIL(0xc0000000084a0f90)) at eval.c:2362
  #59 0x0123b4b4 in Fsetq (args=XIL(0xc0000000084a0f50)) at eval.c:517
  #60 0x01242bf4 in eval_sub (form=XIL(0xc0000000084a0f40)) at eval.c:2239
  #61 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #62 0x0123ae98 in Fcond (args=XIL(0xc0000000084a0df0)) at eval.c:439
  #63 0x01242bf4 in eval_sub (form=XIL(0xc0000000084a0de0)) at eval.c:2239
  #64 0x0123b034 in Fprogn (body=XIL(0xc000000007bfe478)) at eval.c:459
  #65 0x0123e1cd in Flet (args=XIL(0xc000000007bfe458)) at eval.c:973
  #66 0x01242bf4 in eval_sub (form=XIL(0xc000000007bfe448)) at eval.c:2239
  #67 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #68 0x01242bf4 in eval_sub (form=XIL(0xc000000008f21b20)) at eval.c:2239
  #69 0x0123ac55 in Fif (args=XIL(0xc000000008f21b40)) at eval.c:414
  #70 0x01242bf4 in eval_sub (form=XIL(0xc000000008f21b50)) at eval.c:2239
  #71 0x01243ad7 in eval_sub (form=XIL(0xc000000007bfe418)) at eval.c:2358
  #72 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #73 0x01246f3a in funcall_lambda (fun=XIL(0xc000000007bfe3d8), nargs=0,
      arg_vector=0x889d40) at eval.c:3098
  #74 0x012463d3 in apply_lambda (fun=XIL(0xc000000007bfe3c8), args=XIL(0),
      count=63) at eval.c:2959
  #75 0x01243b7b in eval_sub (form=XIL(0xc0000000084853e0)) at eval.c:2362
  #76 0x0123f734 in internal_lisp_condition_case (var=XIL(0x77d5e28),
      bodyform=XIL(0xc0000000084853e0), handlers=XIL(0xc000000008f21a90))
      at eval.c:1307
  #77 0x0123ef97 in Fcondition_case (args=XIL(0xc000000008f21ab0))
      at eval.c:1231
  #78 0x01242bf4 in eval_sub (form=XIL(0xc000000008f21ac0)) at eval.c:2239
  #79 0x01243ad7 in eval_sub (form=XIL(0xc000000008f22690)) at eval.c:2358
  #80 0x01243ad7 in eval_sub (form=XIL(0xc0000000084853b0)) at eval.c:2358
  #81 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #82 0x0123accd in Fif (args=XIL(0xc000000008ff95a0)) at eval.c:415
  #83 0x01242bf4 in eval_sub (form=XIL(0xc000000008ff94e0)) at eval.c:2239
  #84 0x01243ad7 in eval_sub (form=XIL(0xc000000008485370)) at eval.c:2358
  #85 0x0123b034 in Fprogn (body=XIL(0xc000000007c2ff60)) at eval.c:459
  #86 0x0123e1cd in Flet (args=XIL(0xc000000007c2ff40)) at eval.c:973
  #87 0x01242bf4 in eval_sub (form=XIL(0xc000000007c2ff30)) at eval.c:2239
  #88 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #89 0x01246f3a in funcall_lambda (fun=XIL(0xc000000007c2fef0), nargs=1,
      arg_vector=0x88aca0) at eval.c:3098
  #90 0x012463d3 in apply_lambda (fun=XIL(0xc000000007c2fee0),
      args=XIL(0xc000000007d6cbb0), count=53) at eval.c:2959
  #91 0x01243b7b in eval_sub (form=XIL(0xc000000007d6cba0)) at eval.c:2362
  #92 0x0123f734 in internal_lisp_condition_case (var=XIL(0x77d5df0),
      bodyform=XIL(0xc000000007d6cba0), handlers=XIL(0xc000000008ff9710))
      at eval.c:1307
  #93 0x0123ef97 in Fcondition_case (args=XIL(0xc000000008ff96f0))
      at eval.c:1231
  #94 0x01242bf4 in eval_sub (form=XIL(0xc000000008ff96c0)) at eval.c:2239
  #95 0x01243ad7 in eval_sub (form=XIL(0xc000000008ffab30)) at eval.c:2358
  #96 0x01243ad7 in eval_sub (form=XIL(0xc000000007d6cb70)) at eval.c:2358
  #97 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #98 0x01242bf4 in eval_sub (form=XIL(0xc000000008ff3870)) at eval.c:2239
  #99 0x0123ac55 in Fif (args=XIL(0xc000000008ff3850)) at eval.c:414
  #100 0x01242bf4 in eval_sub (form=XIL(0xc000000008ff3840)) at eval.c:2239
  #101 0x01243ad7 in eval_sub (form=XIL(0xc000000007d6cb40)) at eval.c:2358
  #102 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #103 0x0123e1cd in Flet (args=XIL(0xc000000007d6cb10)) at eval.c:973
  #104 0x01242bf4 in eval_sub (form=XIL(0xc000000007d6cb00)) at eval.c:2239
  #105 0x0123b034 in Fprogn (body=XIL(0xc000000007d6c970)) at eval.c:459
  #106 0x01246f3a in funcall_lambda (fun=XIL(0xc000000007d6c900), nargs=1,
      arg_vector=0x88be90) at eval.c:3098
  #107 0x01245840 in Ffuncall (nargs=2, args=0x88be88) at eval.c:2836
  #108 0x012ae87f in exec_byte_code (bytestr=XIL(0x8000000008f6c650),
      vector=XIL(0xa000000008ffd950), maxdepth=make_number(18),
      args_template=make_number(513), nargs=1, args=0x88c620) at bytecode.c:632
  #109 0x01246906 in funcall_lambda (fun=XIL(0xa000000008ffdc90), nargs=1,
      arg_vector=0x88c618) at eval.c:3023
  #110 0x01245686 in Ffuncall (nargs=2, args=0x88c610) at eval.c:2824
  #111 0x012ae87f in exec_byte_code (bytestr=XIL(0x8000000008f30208),
      vector=XIL(0xa000000008fff278), maxdepth=make_number(8),
      args_template=make_number(257), nargs=1, args=0x88cc18) at bytecode.c:632
  #112 0x01246906 in funcall_lambda (fun=XIL(0xa000000008fc1e90), nargs=1,
      arg_vector=0x88cc10) at eval.c:3023
  #113 0x01245686 in Ffuncall (nargs=2, args=0x88cc08) at eval.c:2824
  #114 0x012ae87f in exec_byte_code (bytestr=XIL(0x8000000008f30228),
      vector=XIL(0xa000000008fb80f8), maxdepth=make_number(10),
      args_template=make_number(256), nargs=0, args=0x88d258) at bytecode.c:632
  #115 0x01246906 in funcall_lambda (fun=XIL(0xa000000008fc1e60), nargs=0,
      arg_vector=0x88d258) at eval.c:3023
  #116 0x01245686 in Ffuncall (nargs=1, args=0x88d250) at eval.c:2824
  #117 0x0124304b in eval_sub (form=XIL(0xc0000000080c8e08)) at eval.c:2258
  #118 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #119 0x0123accd in Fif (args=XIL(0xc0000000080c8dd8)) at eval.c:415
  #120 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c8dc8)) at eval.c:2239
  #121 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #122 0x0123ae98 in Fcond (args=XIL(0xc0000000080c6930)) at eval.c:439
  #123 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c6900)) at eval.c:2239
  #124 0x0123b034 in Fprogn (body=XIL(0xc0000000080c6860)) at eval.c:459
  #125 0x0123d8e6 in FletX (args=XIL(0xc0000000080c6820)) at eval.c:904
  #126 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c6810)) at eval.c:2239
  #127 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #128 0x0123b081 in prog_ignore (body=XIL(0xc0000000080c6800)) at eval.c:470
  #129 0x0123e311 in Fwhile (args=XIL(0xc0000000080c67f0)) at eval.c:992
  #130 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c67e0)) at eval.c:2239
  #131 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #132 0x0123d8e6 in FletX (args=XIL(0xc0000000080c67a0)) at eval.c:904
  #133 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c6790)) at eval.c:2239
  #134 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #135 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c6770)) at eval.c:2239
  #136 0x0123ac55 in Fif (args=XIL(0xc0000000080c6750)) at eval.c:414
  #137 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c6740)) at eval.c:2239
  #138 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #139 0x0123e1cd in Flet (args=XIL(0xc0000000080c6720)) at eval.c:973
  #140 0x01242bf4 in eval_sub (form=XIL(0xc0000000080c6710)) at eval.c:2239
  #141 0x0123b034 in Fprogn (body=XIL(0xc0000000080ba3e0)) at eval.c:459
  #142 0x0123e1cd in Flet (args=XIL(0xc0000000080ba3c0)) at eval.c:973
  #143 0x01242bf4 in eval_sub (form=XIL(0xc0000000080ba3b0)) at eval.c:2239
  #144 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #145 0x01246f3a in funcall_lambda (fun=XIL(0xc0000000080ba350), nargs=1,
      arg_vector=0x88e8c0) at eval.c:3098
  #146 0x012463d3 in apply_lambda (fun=XIL(0xc0000000080ba340),
      args=XIL(0xc0000000080ba2d8), count=13) at eval.c:2959
  #147 0x01243b7b in eval_sub (form=XIL(0xc0000000080ba2c8)) at eval.c:2362
  #148 0x0123b034 in Fprogn (body=XIL(0xc0000000080b97a0)) at eval.c:459
  #149 0x01246f3a in funcall_lambda (fun=XIL(0xc0000000080b9600), nargs=0,
      arg_vector=0x88ec10) at eval.c:3098
  #150 0x012463d3 in apply_lambda (fun=XIL(0xc0000000080b95f0), args=XIL(0),
      count=11) at eval.c:2959
  #151 0x01243b7b in eval_sub (form=XIL(0xc0000000080b95a8)) at eval.c:2362
  #152 0x0123eca4 in Funwind_protect (args=XIL(0xc0000000080b8900))
      at eval.c:1195
  #153 0x01242bf4 in eval_sub (form=XIL(0xc0000000080b88f0)) at eval.c:2239
  #154 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #155 0x0123e1cd in Flet (args=XIL(0xc0000000080b88d0)) at eval.c:973
  #156 0x01242bf4 in eval_sub (form=XIL(0xc0000000080b88c0)) at eval.c:2239
  #157 0x0123b034 in Fprogn (body=XIL(0xc0000000080b8320)) at eval.c:459
  #158 0x0123accd in Fif (args=XIL(0xc0000000080b8250)) at eval.c:415
  #159 0x01242bf4 in eval_sub (form=XIL(0xc0000000080b8240)) at eval.c:2239
  #160 0x0123b034 in Fprogn (body=XIL(0)) at eval.c:459
  #161 0x01246f3a in funcall_lambda (fun=XIL(0xc0000000080b8200), nargs=0,
      arg_vector=0x88f5e0) at eval.c:3098
  #162 0x012463d3 in apply_lambda (fun=XIL(0xc0000000080b81f0), args=XIL(0),
      count=4) at eval.c:2959
  #163 0x01243b7b in eval_sub (form=XIL(0xc000000008697260)) at eval.c:2362
  #164 0x01242437 in Feval (form=XIL(0xc000000008697260), lexical=XIL(0))
      at eval.c:2107
  #165 0x011587fa in top_level_2 () at keyboard.c:1122
  #166 0x0123f7d4 in internal_condition_case (bfun=0x11587c4 <top_level_2>,
      handlers=XIL(0x5c50), hfun=0x1157f43 <cmd_error>) at eval.c:1336
  #167 0x0115886f in top_level_1 (ignore=XIL(0)) at keyboard.c:1130
  #168 0x0123e8e5 in internal_catch (tag=XIL(0xf6c0),
      func=0x1158800 <top_level_1>, arg=XIL(0)) at eval.c:1101
  #169 0x011586d0 in command_loop () at keyboard.c:1091
  #170 0x0115797b in recursive_edit_1 () at keyboard.c:698
  #171 0x01157c19 in Frecursive_edit () at keyboard.c:769
  #172 0x011554c0 in main (argc=13, argv=0xcf3610) at emacs.c:1916

  Lisp Backtrace:
  "insert-file-contents" (0x8859c8)
  "let" (0x885cd8)
  "save-current-buffer" (0x885ed8)
  "with-current-buffer" (0x886058)
  "let" (0x886338)
  "unwind-protect" (0x8864f8)
  "let*" (0x886748)
  "if" (0x886948)
  "load-with-code-conversion" (0x886be0)
  "require" (0x8873e8)
  "eval-buffer" (0x887838)
  "let" (0x887b58)
  "unwind-protect" (0x887d18)
  "let*" (0x887f68)
  "if" (0x888168)
  "load-with-code-conversion" (0x888400)
  "require" (0x888c08)
  "dir-locals-read-from-dir" (0x888e10)
  "setq" (0x889198)
  "cond" (0x889398)
  "let" (0x889678)
  "progn" (0x889838)
  "if" (0x8899f8)
  "when" (0x889b78)
  "hack-dir-local-variables" (0x889d40)
  "condition-case" (0x88a188)
  "condition-case-unless-debug" (0x88a308)
  "with-demoted-errors" (0x88a488)
  "if" (0x88a688)
  "unless" (0x88a808)
  "let" (0x88aad8)
  "hack-local-variables" (0x88aca0)
  "condition-case" (0x88b0f8)
  "condition-case-unless-debug" (0x88b278)
  "with-demoted-errors" (0x88b3f8)
  "progn" (0x88b5b8)
  "if" (0x88b778)
  "when" (0x88b8f8)
  "let" (0x88bbc8)
  "normal-mode" (0x88be90)
  "byte-compile-file" (0x88c618)
  "batch-byte-compile-file" (0x88cc10)
  "batch-byte-compile" (0x88d258)
  "funcall" (0x88d250)
  "if" (0x88d508)
  "cond" (0x88d708)
  "let*" (0x88d958)
  "while" (0x88db88)
  "let*" (0x88ddd8)
  "progn" (0x88df98)
  "if" (0x88e158)
  "let" (0x88e428)
  "let" (0x88e6f8)
  "command-line-1" (0x88e8c0)
  "command-line" (0x88ec10)
  "unwind-protect" (0x88ef48)
  "let" (0x88f218)
  "if" (0x88f418)
  "normal-top-level" (0x88f5e0)
  (gdb) fr 4
  #4  0x011acb8b in prepare_to_modify_buffer_1 (start=1, end=1,
      preserve_ptr=0x0) at insdel.c:1937
  1937      eassert (!pdumper_object_p (BEG_ADDR));
  (gdb) p BEG_ADDR
  $4 = (unsigned char *) 0x86d0000 ""
  (gdb) p/x dump_public
  $5 = {
    start = 0x7b80000,
    end = 0x8f1e938
  }

Re: Preview: portable dumper

[Daniel Colascione]

On 02/14/2018 08:24 AM, Eli Zaretskii wrote:
>> From: Robert Pluim <rpluim@gmail.com>
>> Date: Wed, 14 Feb 2018 11:30:07 +0100
>>
>>>> What should we look for?
>>>
>>> The usual stuff, I think: does it build, does it work as expected,
>>> etc.
>>
>> You mean like:
> 
> Yes, like that.
> 
> I see something slightly different on MS-Windows (in a build with
> "--enable-checking"), but maybe similar enough to be explained by the
> same problem:
> 
>         ELC      ../lisp/composite.elc
>       load_dump completed in 46.005 milliseconds
> 
>       insdel.c:1937: Emacs fatal error: assertion failed: !pdumper_object_p (BEG_ADDR)
> 
> Backtrace:
> 
>    #0  0x762c3227 in KERNELBASE!DebugBreak ()
>       from C:\Windows\syswow64\KernelBase.dll
>    #1  0x0131940d in emacs_abort () at w32fns.c:10874
>    #2  0x01152dea in terminate_due_to_signal (sig=22, backtrace_limit=2147483647)
>        at emacs.c:388
>    #3  0x01206274 in die (
>        msg=0x16ce7d2 <DEFAULT_REHASH_SIZE+354> "!pdumper_object_p (BEG_ADDR)",
>        file=0x16ce674 <DEFAULT_REHASH_SIZE+4> "insdel.c", line=1937)
>        at alloc.c:7789
>    #4  0x011acb8b in prepare_to_modify_buffer_1 (start=1, end=1,
>        preserve_ptr=0x0) at insdel.c:1937

It's weird that we're failing there. If we're looking at a buffer with 
dumped contents, we set b->text->beg to NULL, then use the normal 
buffer-allocation procedure (whichever we're compiled to use) to 
allocate memory for the contents. How can the resulting address ever be 
equal to what we started with? Neither mmap_realloc nor r_re_alloc nor 
xrealloc should ever reuse the address.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/14/2018 09:49 AM, Daniel Colascione wrote:
> On 02/14/2018 08:24 AM, Eli Zaretskii wrote:
>>> From: Robert Pluim <rpluim@gmail.com>
>>> Date: Wed, 14 Feb 2018 11:30:07 +0100
>>>
>>>>> What should we look for?
>>>>
>>>> The usual stuff, I think: does it build, does it work as expected,
>>>> etc.
>>>
>>> You mean like:
>>
>> Yes, like that.
>>
>> I see something slightly different on MS-Windows (in a build with
>> "--enable-checking"), but maybe similar enough to be explained by the
>> same problem:
>>
>>         ELC      ../lisp/composite.elc
>>       load_dump completed in 46.005 milliseconds
>>
>>       insdel.c:1937: Emacs fatal error: assertion failed: 
>> !pdumper_object_p (BEG_ADDR)
>>
>> Backtrace:
>>
>>    #0  0x762c3227 in KERNELBASE!DebugBreak ()
>>       from C:\Windows\syswow64\KernelBase.dll
>>    #1  0x0131940d in emacs_abort () at w32fns.c:10874
>>    #2  0x01152dea in terminate_due_to_signal (sig=22, 
>> backtrace_limit=2147483647)
>>        at emacs.c:388
>>    #3  0x01206274 in die (
>>        msg=0x16ce7d2 <DEFAULT_REHASH_SIZE+354> "!pdumper_object_p 
>> (BEG_ADDR)",
>>        file=0x16ce674 <DEFAULT_REHASH_SIZE+4> "insdel.c", line=1937)
>>        at alloc.c:7789
>>    #4  0x011acb8b in prepare_to_modify_buffer_1 (start=1, end=1,
>>        preserve_ptr=0x0) at insdel.c:1937
> 
> It's weird that we're failing there. If we're looking at a buffer with 
> dumped contents, we set b->text->beg to NULL, then use the normal 
> buffer-allocation procedure (whichever we're compiled to use) to 
> allocate memory for the contents. How can the resulting address ever be 
> equal to what we started with? Neither mmap_realloc nor r_re_alloc nor 
> xrealloc should ever reuse the address.

Oh, I think I know what's going on. We must be reallocating from the 
discardable region, which we unmapped on startup, but pdumper_object_p 
still thinks that this memory (which the allocator can legitimately use 
for some other purpose) is part of the dump (since pdumper_object_p uses 
a simple range check) and malfunctions.  I'll add a patch to keep this 
address space reserved.

I just added the actually-discard-the-discardable region code, so I'm 
not surprised it broke. On my machine, we happen to never reuse that 
address space.

Re: Preview: portable dumper

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Daniel Colascione <dancol@dancol.org>
> Date: Wed, 14 Feb 2018 09:49:49 -0800
> 
> >    #3  0x01206274 in die (
> >        msg=0x16ce7d2 <DEFAULT_REHASH_SIZE+354> "!pdumper_object_p (BEG_ADDR)",
> >        file=0x16ce674 <DEFAULT_REHASH_SIZE+4> "insdel.c", line=1937)
> >        at alloc.c:7789
> >    #4  0x011acb8b in prepare_to_modify_buffer_1 (start=1, end=1,
> >        preserve_ptr=0x0) at insdel.c:1937
> 
> It's weird that we're failing there. If we're looking at a buffer with 
> dumped contents, we set b->text->beg to NULL, then use the normal 
> buffer-allocation procedure (whichever we're compiled to use) to 
> allocate memory for the contents. How can the resulting address ever be 
> equal to what we started with? Neither mmap_realloc nor r_re_alloc nor 
> xrealloc should ever reuse the address.

You are talking about what enlarge_buffer_text does?  IOW, this:

    if (pdumper_object_p (old_beg))
      b->text->beg = NULL;
    else
      old_beg = NULL;

  #if defined USE_MMAP_FOR_BUFFERS
    p = mmap_realloc ((void **) &b->text->beg, new_nbytes);
  #elif defined REL_ALLOC
    p = r_re_alloc ((void **) &b->text->beg, new_nbytes);
  #else
    p = xrealloc (b->text->beg, new_nbytes);
  #endif

I will try to look around this when I have time, and see what I find.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/14/2018 11:07 AM, Eli Zaretskii wrote:
>> Cc: emacs-devel@gnu.org
>> From: Daniel Colascione <dancol@dancol.org>
>> Date: Wed, 14 Feb 2018 09:49:49 -0800
>>
>>>     #3  0x01206274 in die (
>>>         msg=0x16ce7d2 <DEFAULT_REHASH_SIZE+354> "!pdumper_object_p (BEG_ADDR)",
>>>         file=0x16ce674 <DEFAULT_REHASH_SIZE+4> "insdel.c", line=1937)
>>>         at alloc.c:7789
>>>     #4  0x011acb8b in prepare_to_modify_buffer_1 (start=1, end=1,
>>>         preserve_ptr=0x0) at insdel.c:1937
>>
>> It's weird that we're failing there. If we're looking at a buffer with
>> dumped contents, we set b->text->beg to NULL, then use the normal
>> buffer-allocation procedure (whichever we're compiled to use) to
>> allocate memory for the contents. How can the resulting address ever be
>> equal to what we started with? Neither mmap_realloc nor r_re_alloc nor
>> xrealloc should ever reuse the address.
> 
> You are talking about what enlarge_buffer_text does?  IOW, this:

It should be fixed now. Give it a shot.

Re: Preview: portable dumper

[Eli Zaretskii]

> Cc: rpluim@gmail.com, emacs-devel@gnu.org
> From: Daniel Colascione <dancol@dancol.org>
> Date: Wed, 14 Feb 2018 11:26:37 -0800
> 
> >> It's weird that we're failing there. If we're looking at a buffer with
> >> dumped contents, we set b->text->beg to NULL, then use the normal
> >> buffer-allocation procedure (whichever we're compiled to use) to
> >> allocate memory for the contents. How can the resulting address ever be
> >> equal to what we started with? Neither mmap_realloc nor r_re_alloc nor
> >> xrealloc should ever reuse the address.
> > 
> > You are talking about what enlarge_buffer_text does?  IOW, this:
> 
> It should be fixed now. Give it a shot.

Thanks, the problem is indeed fixed, and with that I was able to
successfully complete the bootstrap.

I also tested an optimized build, and a parallel "make -j8" build, and
they all worked as expected.

The times to load the .pdmp file are around 20 msec for an unoptimized
build and 7 msec for an optimized build, which I think is very
impressive.

For the record, this is a 32-bit MinGW build configured with
"--with-large-int --with-modules --enable-checking", on a 3-year old
Core i7 box running Windows 7.

Re: Preview: portable dumper

[Andy Moreton]

On Thu 15 Feb 2018, Eli Zaretskii wrote:

>> Cc: rpluim@gmail.com, emacs-devel@gnu.org
>> From: Daniel Colascione <dancol@dancol.org>
>> Date: Wed, 14 Feb 2018 11:26:37 -0800
>> 
>> >> It's weird that we're failing there. If we're looking at a buffer with
>> >> dumped contents, we set b->text->beg to NULL, then use the normal
>> >> buffer-allocation procedure (whichever we're compiled to use) to
>> >> allocate memory for the contents. How can the resulting address ever be
>> >> equal to what we started with? Neither mmap_realloc nor r_re_alloc nor
>> >> xrealloc should ever reuse the address.
>> > 
>> > You are talking about what enlarge_buffer_text does?  IOW, this:
>> 
>> It should be fixed now. Give it a shot.
>
> Thanks, the problem is indeed fixed, and with that I was able to
> successfully complete the bootstrap.
>
> I also tested an optimized build, and a parallel "make -j8" build, and
> they all worked as expected.
>
> The times to load the .pdmp file are around 20 msec for an unoptimized
> build and 7 msec for an optimized build, which I think is very
> impressive.
>
> For the record, this is a 32-bit MinGW build configured with
> "--with-large-int --with-modules --enable-checking", on a 3-year old
> Core i7 box running Windows 7.

It fails to build for 64bit Windows with Mingw64 on msys2:

C:/emacs/git/emacs/pdumper/src/pdumper.c: In function 'dump_read_all':
C:/emacs/git/emacs/pdumper/src/pdumper.c:4784:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
         read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
                                             ^~~~~~~~~~~~~
In file included from C:/emacs/git/emacs/pdumper/src/character.h:27:0,
                 from C:/emacs/git/emacs/pdumper/src/buffer.h:27,
                 from C:/emacs/git/emacs/pdumper/src/pdumper.c:18:
C:/emacs/git/emacs/pdumper/src/pdumper.c: In function 'dump_object_1':
C:/emacs/git/emacs/pdumper/src/lisp.h:206:5: warning: this statement may fall through [-Wimplicit-fallthrough=]
    (suppress_checking || (cond)     \
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     ? (void) 0       \
     ~~~~~~~~~~~~~~~~~~
     : die (# cond, __FILE__, __LINE__))
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/emacs/git/emacs/pdumper/src/pdumper.c:2915:11: note: in expansion of macro 'eassert'
           eassert ("should not be dumping int: is self-representing" && 0);
           ^~~~~~~
C:/emacs/git/emacs/pdumper/src/pdumper.c:2916:9: note: here
         default:
         ^~~~~~~
cc1.exe: some warnings being treated as errors


This appears to be because sys_read() and _read() take an unsigned int
for count rather than size_t. Changing the code to:

read (fd, (char*) buf + bytes_read, (int)(bytes_to_read - bytes_read));

That builds and appears to run, but it would be nicer to get sys_read
etc. to use the proper types.

    AndyM

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Fri, 16 Feb 2018 11:33:35 +0000
> 
> On Thu 15 Feb 2018, Eli Zaretskii wrote:
> 
> C:/emacs/git/emacs/pdumper/src/pdumper.c: In function 'dump_read_all':
> C:/emacs/git/emacs/pdumper/src/pdumper.c:4784:45: error: conversion to 'unsigned int' from 'size_t {aka long long unsigned int}' may alter its value [-Werror=conversion]
>          read (fd, (char*) buf + bytes_read, bytes_to_read - bytes_read);
>                                              ^~~~~~~~~~~~~
> In file included from C:/emacs/git/emacs/pdumper/src/character.h:27:0,
>                  from C:/emacs/git/emacs/pdumper/src/buffer.h:27,
>                  from C:/emacs/git/emacs/pdumper/src/pdumper.c:18:
> C:/emacs/git/emacs/pdumper/src/pdumper.c: In function 'dump_object_1':
> C:/emacs/git/emacs/pdumper/src/lisp.h:206:5: warning: this statement may fall through [-Wimplicit-fallthrough=]
>     (suppress_checking || (cond)     \
>     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>      ? (void) 0       \
>      ~~~~~~~~~~~~~~~~~~
>      : die (# cond, __FILE__, __LINE__))
>      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> C:/emacs/git/emacs/pdumper/src/pdumper.c:2915:11: note: in expansion of macro 'eassert'
>            eassert ("should not be dumping int: is self-representing" && 0);
>            ^~~~~~~
> C:/emacs/git/emacs/pdumper/src/pdumper.c:2916:9: note: here
>          default:
>          ^~~~~~~
> cc1.exe: some warnings being treated as errors
> 
> 
> This appears to be because sys_read() and _read() take an unsigned int
> for count rather than size_t.

Yes.  Angelo already reported this a few days ago.

> Changing the code to:
> 
> read (fd, (char*) buf + bytes_read, (int)(bytes_to_read - bytes_read));

This cannot be the right solution, because you will lose bits in the
cast, when bytes_to_read is greater than INT_MAX.

When Angelo reported this, I said I didn't understand how a similar
code in sysdep.c:emacs_intr_read does compile without a problem,
although it seems to feed a ptrdiff_t value (which should be 64-bit
wide in the 64-bit Windows build, just like size_t, except for the
signedness), similarly to the above.  Can you spot why that works?
Maybe look at the preprocessed source?  Maybe then we will know how to
fix that properly (or maybe we will discover one more subtle bug ;-).

In general, breaking the read into chunks of INT_MAX should work as
well, but I'd like to know first what's going on in emacs_intr_read,
because perhaps there's a more elegant solution we have available.

Re: Preview: portable dumper

[Andy Moreton]

On Fri 16 Feb 2018, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton@gmail.com>
>> This appears to be because sys_read() and _read() take an unsigned int
>> for count rather than size_t.
>
> Yes.  Angelo already reported this a few days ago.
>
>> Changing the code to:
>> 
>> read (fd, (char*) buf + bytes_read, (int)(bytes_to_read - bytes_read));
>
> This cannot be the right solution, because you will lose bits in the
> cast, when bytes_to_read is greater than INT_MAX.

Agreed - it was a quick and dirty hack to get the build to complete.

> When Angelo reported this, I said I didn't understand how a similar
> code in sysdep.c:emacs_intr_read does compile without a problem,
> although it seems to feed a ptrdiff_t value (which should be 64-bit
> wide in the 64-bit Windows build, just like size_t, except for the
> signedness), similarly to the above.  Can you spot why that works?
> Maybe look at the preprocessed source?  Maybe then we will know how to
> fix that properly (or maybe we will discover one more subtle bug ;-).

No idea - I'll take a look over the weekend.

> In general, breaking the read into chunks of INT_MAX should work as
> well, but I'd like to know first what's going on in emacs_intr_read,
> because perhaps there's a more elegant solution we have available.

Given the mismatch between sys_read/_read and read, doesn't the smae
problem exist elsewhere in emacs also ?

    AndyM

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Fri, 16 Feb 2018 16:50:55 +0000
> 
> > In general, breaking the read into chunks of INT_MAX should work as
> > well, but I'd like to know first what's going on in emacs_intr_read,
> > because perhaps there's a more elegant solution we have available.
> 
> Given the mismatch between sys_read/_read and read, doesn't the smae
> problem exist elsewhere in emacs also ?

That other place is emacs_intr_read.

Re: Preview: portable dumper

[Andy Moreton]

On Fri 16 Feb 2018, Eli Zaretskii wrote:
> When Angelo reported this, I said I didn't understand how a similar
> code in sysdep.c:emacs_intr_read does compile without a problem,
> although it seems to feed a ptrdiff_t value (which should be 64-bit
> wide in the 64-bit Windows build, just like size_t, except for the
> signedness), similarly to the above.  Can you spot why that works?

The difference is at the top of pdumper.c:

#ifdef __GNUC__
# pragma GCC diagnostic error "-Wconversion"
# pragma GCC diagnostic error "-Wshadow"

Everything else appears to be built without that warning as error. If I
add similar pragmas to sysdep.c then it also emits a dozen or so
conversion warnings as errors.

    AndyM

Re: Preview: portable dumper

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Fri, 16 Feb 2018 17:48:05 +0000
> 
> On Fri 16 Feb 2018, Eli Zaretskii wrote:
> > When Angelo reported this, I said I didn't understand how a similar
> > code in sysdep.c:emacs_intr_read does compile without a problem,
> > although it seems to feed a ptrdiff_t value (which should be 64-bit
> > wide in the 64-bit Windows build, just like size_t, except for the
> > signedness), similarly to the above.  Can you spot why that works?
> 
> The difference is at the top of pdumper.c:
> 
> #ifdef __GNUC__
> # pragma GCC diagnostic error "-Wconversion"
> # pragma GCC diagnostic error "-Wshadow"
> 
> Everything else appears to be built without that warning as error. If I
> add similar pragmas to sysdep.c then it also emits a dozen or so
> conversion warnings as errors.

Ah, okay.  So I guess we've just uncovered a subtle bug, which affects
emacs_intr_read as well, and perhaps other places.  Thanks.

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 1783 bytes --]

Re: Preview: portable dumper

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

On 02/16/2018 12:43 PM, Daniel Colascione wrote:
> IMHO, we should enable -Wconversion more broadly.

My experience is just the opposite: i.e., that -Wconversion causes more 
trouble than it cures. The pdumper change to INTEGER_TO_CONS is an 
example of trouble. The only reason for that change is to work around a 
compiler bug in GCC that is caused by -Wconversion, a bug that leads to 
a false alarm. I suggest at least the attached patch, which limits the 
damage to pdumper.c instead of letting it spread to other Emacs modules. 
But better yet, I suggest dropping the idea of using -Wconversion even 
on pdumper.c, as it's counterproductive and in high-quality code almost 
inevitably leads to further obfuscation like ALLOW_IMPLICIT_CONVERSION 
and DISALLOW_IMPLICIT_CONVERSION.

PS. Double-parens like that shouldn't be needed in macro bodies, as each 
macro must parenthesize its argument properly anyway.


[-- Attachment #2: 0001-Better-workaround-for-GCC-bug-with-Wconversion.patch --]
[-- Type: text/x-patch, Size: 1851 bytes --]

From 725d6c22f209c96d4ca263ccc173ae77ae7942cf Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@Penguin.CS.UCLA.EDU>
Date: Fri, 16 Feb 2018 13:06:49 -0800
Subject: [PATCH] Better workaround for GCC bug with -Wconversion

* src/lisp.h (INTEGER_TO_CONS): Undo previous change.
* src/pdumper.c (DEFINE_TOLISP_FUNC):
Disable -Wconversion when INTEGER_TO_CONS is used, too.
---
 src/lisp.h    | 6 +++---
 src/pdumper.c | 5 ++++-
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/src/lisp.h b/src/lisp.h
index 45e8a0d791..6d4635f2a7 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3568,9 +3568,9 @@ extern Lisp_Object arithcompare (Lisp_Object num1, Lisp_Object num2,
    itself, or a cons of two or three integers, or if all else fails a float.
    I should not have side effects.  */
 #define INTEGER_TO_CONS(i)					    \
-  (! FIXNUM_OVERFLOW_P ((i))					    \
-   ? make_number ((EMACS_INT) (i))                                      \
-   : EXPR_SIGNED ((i)) ? intbig_to_lisp ((i)) : uintbig_to_lisp ((i)))
+  (! FIXNUM_OVERFLOW_P (i)					    \
+   ? make_number (i)						    \
+   : EXPR_SIGNED (i) ? intbig_to_lisp (i) : uintbig_to_lisp (i))
 extern Lisp_Object intbig_to_lisp (intmax_t);
 extern Lisp_Object uintbig_to_lisp (uintmax_t);
 
diff --git a/src/pdumper.c b/src/pdumper.c
index a05b2c47ce..cc1847356d 100644
--- a/src/pdumper.c
+++ b/src/pdumper.c
@@ -691,7 +691,10 @@ dump_object_self_representing_p (Lisp_Object object)
   static Lisp_Object                 \
   fn (type value)                    \
   {                                  \
-    return INTEGER_TO_CONS (value);  \
+    ALLOW_IMPLICIT_CONVERSION;       \
+    Lisp_Object result = INTEGER_TO_CONS (value); \
+    DISALLOW_IMPLICIT_CONVERSION;    \
+    return result;                   \
   }
 
 DEFINE_FROMLISP_FUNC (intmax_t_from_lisp, intmax_t);
-- 
2.14.3

Re: Preview: portable dumper

[Daniel Colascione]

On 02/16/2018 01:09 PM, Paul Eggert wrote:
> On 02/16/2018 12:43 PM, Daniel Colascione wrote:
>> IMHO, we should enable -Wconversion more broadly.
> 
> My experience is just the opposite: i.e., that -Wconversion causes more 
> trouble than it cures. The pdumper change to INTEGER_TO_CONS is an 
> example of trouble. The only reason for that change is to work around a 
> compiler bug in GCC that is caused by -Wconversion, a bug that leads to 
> a false alarm. I suggest at least the attached patch, which limits the 
> damage to pdumper.c instead of letting it spread to other Emacs modules. 
> But better yet, I suggest dropping the idea of using -Wconversion even 
> on pdumper.c, as it's counterproductive and in high-quality code almost 
> inevitably leads to further obfuscation like ALLOW_IMPLICIT_CONVERSION 
> and DISALLOW_IMPLICIT_CONVERSION.

-Wconversion helped me find real bugs and otherwise mysterious bugs in 
pdumper. It also, apparently, found a longstanding subtle bug in a 
completely different part of Emacs. Outside of certain specialized 
casting facilities, -Wconversion ought to warn only on things that are 
actually problems. Have a counter-example?

The ALLOW_IMPLICIT_CONVERSION and DISALLOW_IMPLICIT_CONVERSION macros 
are there in pdumper to account for the rest of Emacs not being 
conversion-clean; if everything worked with -Wconversion, we wouldn't 
need the bracketing.

> PS. Double-parens like that shouldn't be needed in macro bodies, as each 
> macro must parenthesize its argument properly anyway.

Thanks.

Re: Preview: portable dumper

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 2585 bytes --]

On 02/16/2018 01:23 PM, Daniel Colascione wrote:
> Outside of certain specialized casting facilities
DEFINE_TOLISP_FUNC is exactly the sort of specialized casting facility 
where -Wconversion is more likely to mess up than usual, which is why I 
suggested disabling -Wconversion there in pdumper.c. -Wconversion is not 
as useful elsewhere, which is why I suggested leaving INTEGER_TO_CONS alone.

> , -Wconversion ought to warn only on things that are actually 
> problems. Have a counter-example

Sure, revert the pdumper change to INTEGER_TO_CONS but leave pdumper.c 
alone, by applying the attached patch to pdumper. The compile on a 
platform where EMACS_INT is 32 bits, using GCC 7.3.1 20180130 (Red Hat 
7.3.1-2). The resulting diagnostic is a false alarm:

pdumper.c: In function ‘intmax_t_to_lisp’:
pdumper.c:694:29: error: conversion to ‘EMACS_INT {aka int}’ from 
‘intmax_t {aka long long int}’ may alter its value [-Werror=conversion]
      return INTEGER_TO_CONS (value);  \
                              ^
lisp.h:3572:19: note: in definition of macro ‘INTEGER_TO_CONS’
     ? make_number (i)          \
                    ^
pdumper.c:698:1: note: in expansion of macro ‘DEFINE_TOLISP_FUNC’
  DEFINE_TOLISP_FUNC (intmax_t_to_lisp, intmax_t);
  ^~~~~~~~~~~~~~~~~~

This false alarm is due to a GCC bug. GCC is supposed to take 
expressions like this one (adapted from INTEGER_TO_CONS):

   (MOST_NEGATIVE_FIXNUM <= i && i <= MOST_POSITIVE_FIXNUM
    ? make_number (i)
    : something_else (i))

and evaluate make_number (i) in the context of i being in range for 
fixnums (which means i is in range for EMACS_INT). In this particular 
case, GCC has a bug where it forgets i's range, and thus the false alarm.

I'd file a bug report with the GCC folks, but in the past my experience 
is that they don't take -Wconversion bug reports all that seriously. In 
practice, -Wconversion isn't good enough for high-quality code like what 
Emacs should be.


> -Wconversion helped me find real bugs and otherwise mysterious bugs in 
> pdumper.
Yes, -Wconversion can find bugs, particularly the first time one writes 
a program when the program has lots of bugs that need finding. But its 
false-alarm rate is too high to be useful in high-quality code. Although 
it's OK to use -Wconversion temporarily in new or otherwise-buggy areas, 
we shouldn't encourage its use in the part of Emacs that is intended to 
be stable and high-quality.


[-- Attachment #2: pdumper.diff --]
[-- Type: text/x-patch, Size: 776 bytes --]

diff --git a/src/lisp.h b/src/lisp.h
index 45e8a0d791..6d4635f2a7 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3568,9 +3568,9 @@ extern Lisp_Object arithcompare (Lisp_Object num1, Lisp_Object num2,
    itself, or a cons of two or three integers, or if all else fails a float.
    I should not have side effects.  */
 #define INTEGER_TO_CONS(i)					    \
-  (! FIXNUM_OVERFLOW_P ((i))					    \
-   ? make_number ((EMACS_INT) (i))                                      \
-   : EXPR_SIGNED ((i)) ? intbig_to_lisp ((i)) : uintbig_to_lisp ((i)))
+  (! FIXNUM_OVERFLOW_P (i)					    \
+   ? make_number (i)						    \
+   : EXPR_SIGNED (i) ? intbig_to_lisp (i) : uintbig_to_lisp (i))
 extern Lisp_Object intbig_to_lisp (intmax_t);
 extern Lisp_Object uintbig_to_lisp (uintmax_t);
 

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 4440 bytes --]

Re: Preview: portable dumper

[Paul Eggert]

On 02/16/2018 02:02 PM, Daniel Colascione wrote:
> I'm not sure what you mean by high quality in this context.

I mean code that is written with reasonable care to check for ranges, 
much as Emacs normally does. For example, INTEGER_TO_CONS checks for 
ranges before converting its argument to EMACS_INT. When such code runs 
afoul of -Wconversion, my experience is that the false alarm rate is so 
high that -Wconversion causes more trouble than it's worth. I suspect 
that pdumper.c also had a bunch of false alarms, which is why it 
contains all that gorp with ALLOW_IMPLICIT_CONVERSION and 
DISALLOW_IMPLICIT_CONVERSION. I'll try to pry free some time to look in 
more detail at pdumper.c and see whether there's a way to fix it without 
worrying so much about -Wconversion.

>     if programmers were angels, no warnings would be necessary. The
>     point is to *keep* the code high quality,
>
Which is why I advocate not using -Wconversion: as a rule, -Wconversion 
takes good code and makes it worse. Of course there are always 
exceptions, and perhaps pdumper.c is one of those exceptions.

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

> On 02/14/2018 11:07 AM, Eli Zaretskii wrote:
>>> Cc: emacs-devel@gnu.org
>>> From: Daniel Colascione <dancol@dancol.org>
>>> Date: Wed, 14 Feb 2018 09:49:49 -0800
>>>
>>>>     #3  0x01206274 in die (
>>>>         msg=0x16ce7d2 <DEFAULT_REHASH_SIZE+354> "!pdumper_object_p (BEG_ADDR)",
>>>>         file=0x16ce674 <DEFAULT_REHASH_SIZE+4> "insdel.c", line=1937)
>>>>         at alloc.c:7789
>>>>     #4  0x011acb8b in prepare_to_modify_buffer_1 (start=1, end=1,
>>>>         preserve_ptr=0x0) at insdel.c:1937
>>>
>>> It's weird that we're failing there. If we're looking at a buffer with
>>> dumped contents, we set b->text->beg to NULL, then use the normal
>>> buffer-allocation procedure (whichever we're compiled to use) to
>>> allocate memory for the contents. How can the resulting address ever be
>>> equal to what we started with? Neither mmap_realloc nor r_re_alloc nor
>>> xrealloc should ever reuse the address.
>>
>> You are talking about what enlarge_buffer_text does?  IOW, this:
>
> It should be fixed now. Give it a shot.

I'm not sure if this was meant for me as well. In any case, I just
pulled and rebuilt, and everything seems to be working well as of
commit 9484bb3ab8e39add474400e5982802b61c56eb3a (I'm using it to write
this message).

Robert

Re: Preview: portable dumper

[Robert Pluim]

Robert Pluim <rpluim@gmail.com> writes:
> I'm not sure if this was meant for me as well. In any case, I just
> pulled and rebuilt, and everything seems to be working well as of
> commit 9484bb3ab8e39add474400e5982802b61c56eb3a (I'm using it to write
> this message).

I spoke too soon. Commit fca8372020248318f1d0007378607a912b19026e is causing

./temacs --batch  --load loadup pbootstrap

to segfault. As far as I can tell, all that commit does is replace
eassert with eassume, but reverting it causes the crash to go away.

$ gcc --version
gcc-5.real (Ubuntu 5.4.0-6ubuntu1~16.04.6) 5.4.0 20160609

Thread 1 "temacs" received signal SIGSEGV, Segmentation fault.
dump_fwd (ctx=0x7fffffffd020, fwd=0x0) at pdumper.c:2277
2277	  switch (XFWDTYPE (fwd))
(gdb) bt
#0  dump_fwd (ctx=0x7fffffffd020, fwd=0x0) at pdumper.c:2277
#1  0x0000000000559be5 in dump_pre_dump_symbol (symbol=0xbf9040 <lispsym>, ctx=0x7fffffffd020) at pdumper.c:2361
#2  dump_hot_parts_of_discardable_objects (ctx=0x7fffffffd020) at pdumper.c:3214
#3  Fdump_emacs_portable (filename=<optimized out>, track_referrers=<optimized out>) at pdumper.c:3909
#4  0x000000000057184c in eval_sub (form=<optimized out>) at eval.c:2293
#5  0x00000000005718a3 in eval_sub (form=<optimized out>) at eval.c:2242
#6  0x0000000000575228 in Fprogn (body=<optimized out>) at eval.c:459
#7  Flet (args=XIL(0xcbb513)) at eval.c:973
#8  0x00000000005718a3 in eval_sub (form=<optimized out>) at eval.c:2242
#9  0x00000000005718a3 in eval_sub (form=form@entry=XIL(0xcbc6e3)) at eval.c:2242
#10 0x0000000000597953 in readevalloop (readcharfun=readcharfun@entry=XIL(0x6a50), 
    infile0=infile0@entry=0x7fffffffd6e0, sourcename=sourcename@entry=XIL(0xcac744), printflag=printflag@entry=false, 
    unibyte=unibyte@entry=XIL(0), readfun=readfun@entry=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2073
#11 0x00000000005980fc in Fload (file=XIL(0xcac624), noerror=<optimized out>, nomessage=XIL(0), 
    nosuffix=<optimized out>, must_suffix=<optimized out>) at lread.c:1457
#12 0x0000000000571810 in eval_sub (form=form@entry=XIL(0xca8e73)) at eval.c:2304
#13 0x00000000005757f8 in Feval (form=XIL(0xca8e73), lexical=<optimized out>) at eval.c:2110
#14 0x000000000057088e in internal_condition_case (bfun=bfun@entry=0x4f53a0 <top_level_2>, 
    handlers=handlers@entry=XIL(0x53d0), hfun=hfun@entry=0x4fabd0 <cmd_error>) at eval.c:1336
#15 0x00000000004f7bfc in top_level_1 (ignore=ignore@entry=XIL(0)) at keyboard.c:1130
#16 0x000000000057082c in internal_catch (tag=tag@entry=XIL(0xcae0), func=func@entry=0x4f7ba0 <top_level_1>, 
    arg=arg@entry=XIL(0)) at eval.c:1101
#17 0x00000000004f5338 in command_loop () at keyboard.c:1091
#18 0x00000000004fa7b6 in recursive_edit_1 () at keyboard.c:698
#19 0x00000000004fab00 in Frecursive_edit () at keyboard.c:769
#20 0x000000000041b015 in main (argc=5, argv=0x7fffffffdaf8) at emacs.c:1929

Lisp Backtrace:
"dump-emacs-portable" (0xffffd200)
"if" (0xffffd2b8)
"let" (0xffffd418)
"if" (0xffffd4d8)
"load" (0xffffd7f0)

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 1145 bytes --]

Re: Preview: portable dumper

[Paul Eggert]

On 02/20/2018 09:19 AM, Daniel Colascione wrote:
> Do you get an assertion failure if you enable assertion checking? If 
> we're seeing this kind of problem from the compiler expectation alone, 
> I'll definitely revert that change.

In my experience, eassume (and 'assume') shouldn't be used for 
complicated expressions, as GCC is too-easily confused by them. eassume 
should be used only for expressions where the assumption really does 
help the compiler, either by generating significantly-better code or by 
pacifying a false alarm.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/20/2018 09:28 AM, Paul Eggert wrote:
> On 02/20/2018 09:19 AM, Daniel Colascione wrote:
>> Do you get an assertion failure if you enable assertion checking? If 
>> we're seeing this kind of problem from the compiler expectation alone, 
>> I'll definitely revert that change.
> 
> In my experience, eassume (and 'assume') shouldn't be used for 
> complicated expressions, as GCC is too-easily confused by them. eassume 
> should be used only for expressions where the assumption really does 
> help the compiler, either by generating significantly-better code or by 
> pacifying a false alarm.

I'd hoped things had gotten better, and some local testing seemed to 
suggest it was safe to use these days. Apparently it's not, which is a 
shame. It'd be nice, but not urgent, to track down which specific 
assumptions caused what I think must be miscompilations so we could 
report them upstream.

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

> On 02/20/2018 09:28 AM, Paul Eggert wrote:
>> On 02/20/2018 09:19 AM, Daniel Colascione wrote:
>>> Do you get an assertion failure if you enable assertion checking?
>>> If we're seeing this kind of problem from the compiler expectation
>>> alone, I'll definitely revert that change.
>>
>> In my experience, eassume (and 'assume') shouldn't be used for
>> complicated expressions, as GCC is too-easily confused by
>> them. eassume should be used only for expressions where the
>> assumption really does help the compiler, either by generating
>> significantly-better code or by pacifying a false alarm.
>
> I'd hoped things had gotten better, and some local testing seemed to
> suggest it was safe to use these days. Apparently it's not, which is a
> shame. It'd be nice, but not urgent, to track down which specific
> assumptions caused what I think must be miscompilations so we could
> report them upstream.

The guilty change is:

--- a/src/pdumper.c
+++ b/src/pdumper.c
@@ -607,7 +607,7 @@ dump_set_have_current_referrer (struct dump_context *ctx, bool have)
 static void
 DUMP_CLEAR_REFERRER (struct dump_context *ctx)
 {
-  eassert (ctx->have_current_referrer);
+  eassume (ctx->have_current_referrer);
   dump_set_have_current_referrer (ctx, false);
   if (dump_tracking_referrers_p (ctx))
     ctx->current_referrer = Qnil;

If I revert just that hunk, the build succeeds.

Robert

Re: Preview: portable dumper

[Daniel Colascione]

On 02/20/2018 10:09 AM, Robert Pluim wrote:
> Daniel Colascione <dancol@dancol.org> writes:
> 
>> On 02/20/2018 09:28 AM, Paul Eggert wrote:
>>> On 02/20/2018 09:19 AM, Daniel Colascione wrote:
>>>> Do you get an assertion failure if you enable assertion checking?
>>>> If we're seeing this kind of problem from the compiler expectation
>>>> alone, I'll definitely revert that change.
>>>
>>> In my experience, eassume (and 'assume') shouldn't be used for
>>> complicated expressions, as GCC is too-easily confused by
>>> them. eassume should be used only for expressions where the
>>> assumption really does help the compiler, either by generating
>>> significantly-better code or by pacifying a false alarm.
>>
>> I'd hoped things had gotten better, and some local testing seemed to
>> suggest it was safe to use these days. Apparently it's not, which is a
>> shame. It'd be nice, but not urgent, to track down which specific
>> assumptions caused what I think must be miscompilations so we could
>> report them upstream.
> 
> The guilty change is:
> 
> --- a/src/pdumper.c
> +++ b/src/pdumper.c
> @@ -607,7 +607,7 @@ dump_set_have_current_referrer (struct dump_context *ctx, bool have)
>   static void
>   DUMP_CLEAR_REFERRER (struct dump_context *ctx)
>   {
> -  eassert (ctx->have_current_referrer);
> +  eassume (ctx->have_current_referrer);
>     dump_set_have_current_referrer (ctx, false);
>     if (dump_tracking_referrers_p (ctx))
>       ctx->current_referrer = Qnil;
> 
> If I revert just that hunk, the build succeeds.

Wow: thanks for tracking that down.

What on earth? I mean, we're instructing the compiler to assume 
something that's both trivial and true. What GCC version is this? If 
it's recent, I think it's probably worth reporting.

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

> On 02/20/2018 10:09 AM, Robert Pluim wrote:
>> Daniel Colascione <dancol@dancol.org> writes:
>>
>>> On 02/20/2018 09:28 AM, Paul Eggert wrote:
>>>> On 02/20/2018 09:19 AM, Daniel Colascione wrote:
>>>>> Do you get an assertion failure if you enable assertion checking?
>>>>> If we're seeing this kind of problem from the compiler expectation
>>>>> alone, I'll definitely revert that change.
>>>>
>>>> In my experience, eassume (and 'assume') shouldn't be used for
>>>> complicated expressions, as GCC is too-easily confused by
>>>> them. eassume should be used only for expressions where the
>>>> assumption really does help the compiler, either by generating
>>>> significantly-better code or by pacifying a false alarm.
>>>
>>> I'd hoped things had gotten better, and some local testing seemed to
>>> suggest it was safe to use these days. Apparently it's not, which is a
>>> shame. It'd be nice, but not urgent, to track down which specific
>>> assumptions caused what I think must be miscompilations so we could
>>> report them upstream.
>>
>> The guilty change is:
>>
>> --- a/src/pdumper.c
>> +++ b/src/pdumper.c
>> @@ -607,7 +607,7 @@ dump_set_have_current_referrer (struct dump_context *ctx, bool have)
>>   static void
>>   DUMP_CLEAR_REFERRER (struct dump_context *ctx)
>>   {
>> -  eassert (ctx->have_current_referrer);
>> +  eassume (ctx->have_current_referrer);
>>     dump_set_have_current_referrer (ctx, false);
>>     if (dump_tracking_referrers_p (ctx))
>>       ctx->current_referrer = Qnil;
>>
>> If I revert just that hunk, the build succeeds.
>
> Wow: thanks for tracking that down.
>

splitpatch + git bisect run can do wonders :-)

> What on earth? I mean, we're instructing the compiler to assume
> something that's both trivial and true. What GCC version is this? If
> it's recent, I think it's probably worth reporting.

$ gcc --version
gcc-5.real (Ubuntu 5.4.0-6ubuntu1~16.04.6) 5.4.0 20160609

I have a gcc-8-ish somewhere that I can rev up to see if it has the
same problem, but probably not soon.

Robert

Re: Preview: portable dumper

[Robert Pluim]

Robert Pluim <rpluim@gmail.com> writes:

> Daniel Colascione <dancol@dancol.org> writes:
>
>> On 02/20/2018 10:09 AM, Robert Pluim wrote:
>> What on earth? I mean, we're instructing the compiler to assume
>> something that's both trivial and true. What GCC version is this? If
>> it's recent, I think it's probably worth reporting.
>
> $ gcc --version
> gcc-5.real (Ubuntu 5.4.0-6ubuntu1~16.04.6) 5.4.0 20160609
>
> I have a gcc-8-ish somewhere that I can rev up to see if it has the
> same problem, but probably not soon.

That gcc-8 turned out to be easy to get ready, and it shows the same
problem (and solution). So now I have an alpha branch of emacs being
allegedly miscompiled by a non-released tip-of-tree version of gcc, in
a feature that messes around with the guts of Emacs' object
representations.

I find it hard to assign blame here. Should I start using unreleased
binutils next? :-)

Robert

Re: Preview: portable dumper

[Stefan Monnier]

> I find it hard to assign blame here.

Really ?!?
Isn't it just always the other guy?


        Stefan

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

> On Feb 20, 2018 8:37 AM, Robert Pluim <rpluim@gmail.com> wrote:
>
>  Robert Pluim <rpluim@gmail.com> writes: 
>  > I'm not sure if this was meant for me as well. In any case, I just 
>  > pulled and rebuilt, and everything seems to be working well as of 
>  > commit 9484bb3ab8e39add474400e5982802b61c56eb3a (I'm using it to write 
>  > this message). 
>
>  I spoke too soon. Commit fca8372020248318f1d0007378607a912b19026e is causing 
>
>  ./temacs --batch  --load loadup pbootstrap 
>
>  to segfault. As far as I can tell, all that commit does is replace 
>  eassert with eassume, but reverting it causes the crash to go away. 
>
> Do you get an assertion failure if you enable assertion checking? If we're seeing this kind of problem from the compiler expectation alone, I'll definitely revert that change.

No, --enable-checking=all gets me a different segfault:

(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00000000005d4556 in unbind_to (count=<optimized out>, value=value@entry=XIL(0)) at eval.c:3576
#2  0x000000000061bb01 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, 
    args_template=args_template@entry=XIL(0), nargs=nargs@entry=0, args=<optimized out>, args@entry=0x0)
    at bytecode.c:652
#3  0x00000000005d5538 in funcall_lambda (fun=XIL(0xe005c5), nargs=nargs@entry=3, 
    arg_vector=arg_vector@entry=0x7fffffffccb0) at eval.c:3108
#4  0x00000000005d5983 in Ffuncall (nargs=4, args=args@entry=0x7fffffffcca8) at eval.c:2839
#5  0x000000000061bb88 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, 
    args_template=args_template@entry=XIL(0), nargs=nargs@entry=0, args=<optimized out>, args@entry=0x0)
    at bytecode.c:632
#6  0x00000000005d5538 in funcall_lambda (fun=XIL(0xe004a5), nargs=nargs@entry=4, 
    arg_vector=arg_vector@entry=0x7fffffffcec0) at eval.c:3108
#7  0x00000000005d5983 in Ffuncall (nargs=5, args=args@entry=0x7fffffffceb8) at eval.c:2839
#8  0x000000000061bb88 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, 
    args_template=args_template@entry=XIL(0), nargs=nargs@entry=0, args=<optimized out>, args@entry=0x0)
    at bytecode.c:632
#9  0x00000000005d5538 in funcall_lambda (fun=fun@entry=XIL(0xdfc155), nargs=nargs@entry=9, 
    arg_vector=arg_vector@entry=0x7fffffffd160) at eval.c:3108
#10 0x00000000005d4765 in apply_lambda (fun=XIL(0xdfc155), args=<optimized out>, count=count@entry=30) at eval.c:2962
#11 0x00000000005d4b42 in eval_sub (form=form@entry=XIL(0xdf94a3)) at eval.c:2365
#12 0x0000000000607604 in readevalloop (readcharfun=readcharfun@entry=XIL(0x6a50), 
    infile0=infile0@entry=0x7fffffffd480, sourcename=sourcename@entry=XIL(0xe37814), printflag=printflag@entry=false, 
    unibyte=unibyte@entry=XIL(0), readfun=readfun@entry=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2073
#13 0x0000000000607d9b in Fload (file=XIL(0xe37894), noerror=<optimized out>, nomessage=XIL(0), 
    nosuffix=<optimized out>, must_suffix=<optimized out>) at lread.c:1457
#14 0x00000000005d4d64 in eval_sub (form=form@entry=XIL(0xd99ea3)) at eval.c:2304
#15 0x0000000000607604 in readevalloop (readcharfun=readcharfun@entry=XIL(0x6a50), 
    infile0=infile0@entry=0x7fffffffd7a0, sourcename=sourcename@entry=XIL(0xde5394), printflag=printflag@entry=false, 
    unibyte=unibyte@entry=XIL(0), readfun=readfun@entry=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2073
#16 0x0000000000607d9b in Fload (file=XIL(0xde5274), noerror=<optimized out>, nomessage=XIL(0), 
    nosuffix=<optimized out>, must_suffix=<optimized out>) at lread.c:1457
#17 0x00000000005d4d64 in eval_sub (form=form@entry=XIL(0xde2aa3)) at eval.c:2304
#18 0x00000000005d9db8 in Feval (form=XIL(0xde2aa3), lexical=<optimized out>) at eval.c:2110
#19 0x00000000005d3be7 in internal_condition_case (bfun=bfun@entry=0x53b320 <top_level_2>, 
    handlers=handlers@entry=XIL(0x53d0), hfun=hfun@entry=0x541d80 <cmd_error>) at eval.c:1336
#20 0x000000000053ee4c in top_level_1 (ignore=ignore@entry=XIL(0)) at keyboard.c:1130
#21 0x00000000005d3b2c in internal_catch (tag=tag@entry=XIL(0xcae0), func=func@entry=0x53edf0 <top_level_1>, 
    arg=arg@entry=XIL(0)) at eval.c:1101
#22 0x000000000053b2b8 in command_loop () at keyboard.c:1091
#23 0x00000000005418b6 in recursive_edit_1 () at keyboard.c:698
#24 0x0000000000541c38 in Frecursive_edit () at keyboard.c:769
#25 0x000000000041c197 in main (argc=5, argv=0x7fffffffdbb8) at emacs.c:1929
(gdb) 

Re: Preview: portable dumper

[Robert Pluim]

Robert Pluim <rpluim@gmail.com> writes:

>> Do you get an assertion failure if you enable assertion checking? If we're seeing this kind of problem from the compiler expectation alone, I'll definitely revert that change.
>
> No, --enable-checking=all gets me a different segfault:

And in fact I get that segfault even with
fca8372020248318f1d0007378607a912b19026e reverted, but only with
--enable-checking=all

Robert

Re: Preview: portable dumper

[Daniel Colascione]

On 02/20/2018 09:45 AM, Robert Pluim wrote:
> Robert Pluim <rpluim@gmail.com> writes:
> 
>>> Do you get an assertion failure if you enable assertion checking? If we're seeing this kind of problem from the compiler expectation alone, I'll definitely revert that change.
>>
>> No, --enable-checking=all gets me a different segfault:
> 
> And in fact I get that segfault even with
> fca8372020248318f1d0007378607a912b19026e reverted, but only with
> --enable-checking=all

Can you repro with an -O0 build?

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

> On 02/20/2018 09:45 AM, Robert Pluim wrote:
>> Robert Pluim <rpluim@gmail.com> writes:
>>
>>>> Do you get an assertion failure if you enable assertion checking? If we're seeing this kind of problem from the compiler expectation alone, I'll definitely revert that change.
>>>
>>> No, --enable-checking=all gets me a different segfault:
>>
>> And in fact I get that segfault even with
>> fca8372020248318f1d0007378607a912b19026e reverted, but only with
>> --enable-checking=all
>
> Can you repro with an -O0 build?

Yes. Backtrace:

(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x000000000066dd30 in do_one_unbind (this_binding=0x7fffffffbf40, unwinding=true, bindflag=SET_INTERNAL_UNBIND)
    at eval.c:3457
#2  0x000000000066e17e in unbind_to (count=48, value=XIL(0)) at eval.c:3576
#3  0x00000000006be902 in exec_byte_code (bytestr=XIL(0xf1dbf4), vector=XIL(0xef3585), maxdepth=make_number(5), 
    args_template=XIL(0), nargs=0, args=0x0) at bytecode.c:652
#4  0x000000000066cce0 in funcall_lambda (fun=XIL(0xf015c5), nargs=3, arg_vector=0xef3585) at eval.c:3108
#5  0x000000000066bbf8 in Ffuncall (nargs=4, args=0x7fffffffc438) at eval.c:2827
#6  0x00000000006be824 in exec_byte_code (bytestr=XIL(0xf178e4), vector=XIL(0xf2fca5), maxdepth=make_number(4), 
    args_template=XIL(0), nargs=0, args=0x0) at bytecode.c:632
#7  0x000000000066cce0 in funcall_lambda (fun=XIL(0xf014a5), nargs=4, arg_vector=0xf2fca5) at eval.c:3108
#8  0x000000000066bbf8 in Ffuncall (nargs=5, args=0x7fffffffc8e8) at eval.c:2827
#9  0x00000000006be824 in exec_byte_code (bytestr=XIL(0xef7444), vector=XIL(0xf0d6c5), maxdepth=make_number(6), 
    args_template=XIL(0), nargs=0, args=0x0) at bytecode.c:632
#10 0x000000000066cce0 in funcall_lambda (fun=XIL(0xefd155), nargs=9, arg_vector=0xf0d6c5) at eval.c:3108
#11 0x000000000066c4d1 in apply_lambda (fun=XIL(0xefd155), args=XIL(0xefa303), count=30) at eval.c:2962
#12 0x000000000066a40d in eval_sub (form=XIL(0xefa483)) at eval.c:2335
#13 0x00000000006a3627 in readevalloop (readcharfun=XIL(0x6a50), infile0=0x7fffffffd210, sourcename=XIL(0xf38814), 
    printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2073
#14 0x00000000006a17ce in Fload (file=XIL(0xf38894), noerror=XIL(0), nomessage=XIL(0), nosuffix=XIL(0), 
    must_suffix=XIL(0)) at lread.c:1457
#15 0x000000000066a2e4 in eval_sub (form=XIL(0xe9ae83)) at eval.c:2304
#16 0x00000000006a3627 in readevalloop (readcharfun=XIL(0x6a50), infile0=0x7fffffffd5f0, sourcename=XIL(0xee6394), 
    printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2073
#17 0x00000000006a17ce in Fload (file=XIL(0xee6274), noerror=XIL(0), nomessage=XIL(0), nosuffix=XIL(0), 
    must_suffix=XIL(0)) at lread.c:1457
#18 0x000000000066a2e4 in eval_sub (form=XIL(0xee3aa3)) at eval.c:2304
#19 0x00000000006696e0 in Feval (form=XIL(0xee3aa3), lexical=XIL(0)) at eval.c:2110
#20 0x000000000059ed23 in top_level_2 () at keyboard.c:1122
#21 0x0000000000667743 in internal_condition_case (bfun=0x59ed00 <top_level_2>, handlers=XIL(0x53d0), 
    hfun=0x59e6ef <cmd_error>) at eval.c:1336
#22 0x000000000059ed67 in top_level_1 (ignore=XIL(0)) at keyboard.c:1130
#23 0x0000000000666c04 in internal_catch (tag=XIL(0xcae0), func=0x59ed25 <top_level_1>, arg=XIL(0)) at eval.c:1101
#24 0x000000000059ec52 in command_loop () at keyboard.c:1091
#25 0x000000000059e1d4 in recursive_edit_1 () at keyboard.c:698
#26 0x000000000059e3ce in Frecursive_edit () at keyboard.c:769
#27 0x000000000059c223 in main (argc=5, argv=0x7fffffffdbb8) at emacs.c:1929
(gdb) up
#1  0x000000000066dd30 in do_one_unbind (this_binding=0x7fffffffbf40, unwinding=true, bindflag=SET_INTERNAL_UNBIND)
    at eval.c:3457
3457	      this_binding->unwind.func (this_binding->unwind.arg);
(gdb) p this_binding->unwind.arg
$1 = XIL(0)

Re: Preview: portable dumper

[Daniel Colascione]

On 02/20/2018 09:45 AM, Robert Pluim wrote:
> Robert Pluim <rpluim@gmail.com> writes:
> 
>>> Do you get an assertion failure if you enable assertion checking? If we're seeing this kind of problem from the compiler expectation alone, I'll definitely revert that change.
>>
>> No, --enable-checking=all gets me a different segfault:
> 
> And in fact I get that segfault even with
> fca8372020248318f1d0007378607a912b19026e reverted, but only with
> --enable-checking=all

Thanks. Can repro locally now.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/20/2018 10:26 AM, Daniel Colascione wrote:
> On 02/20/2018 09:45 AM, Robert Pluim wrote:
>> Robert Pluim <rpluim@gmail.com> writes:
>>
>>>> Do you get an assertion failure if you enable assertion checking? If 
>>>> we're seeing this kind of problem from the compiler expectation 
>>>> alone, I'll definitely revert that change.
>>>
>>> No, --enable-checking=all gets me a different segfault:
>>
>> And in fact I get that segfault even with
>> fca8372020248318f1d0007378607a912b19026e reverted, but only with
>> --enable-checking=all
> 
> Thanks. Can repro locally now.

I wonder whether the --enable-checking=all code is just busted somehow. 
Emacs doesn't finish dumping itself in master either: loadup seems to 
spin after trying to load files.el, even with -O3, and on the pdumper 
branch, we crash in temacs during early loadup well before we do 
anything with pdumper. I'll keep looking when I can.

Re: Preview: portable dumper

[Alan Third]

On Wed, Feb 14, 2018 at 06:29:54AM +0200, Eli Zaretskii wrote:
> > Date: Tue, 13 Feb 2018 23:06:15 +0100 (CET)
> > From: Angelo Graziosi <angelo.g0@libero.it>
> > 
> > Eli Zaretskii wrote:
> > >
> > > I'd urge people to try this branch and report any issues they see.
> > 
> > What should we look for?
> 
> The usual stuff, I think: does it build, does it work as expected,
> etc.

Lots of errors on macOS 10.13 (clang). Looks like they’re mostly the
same thing repeated.

dispnew.c:3019:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("redraw-frame", Fredraw_frame, Sredraw_frame, 0, 1, 0,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:3028:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("redraw-display", Fredraw_display, Sredraw_display, 0, 0, "",
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:5596:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("open-termscript", Fopen_termscript, Sopen_termscript,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:5629:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("send-string-to-terminal", Fsend_string_to_terminal,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:5672:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("ding", Fding, Sding, 0, 1, 0,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:5712:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("sleep-for", Fsleep_for, Ssleep_for, 1, 2, 0,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:5812:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("redisplay", Fredisplay, Sredisplay, 0, 1, 0,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:5852:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("frame-or-buffer-changed-p", Fframe_or_buffer_changed_p,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:6069:34: error: use of undeclared identifier 'will_dump'; did you mean
      'will_dump_p'?
  if (!inhibit_window_system && !will_dump)
                                 ^~~~~~~~~
                                 will_dump_p
./lisp.h:654:1: note: 'will_dump_p' declared here
will_dump_p (void)
^
dispnew.c:6069:34: warning: address of function 'will_dump_p' will always
      evaluate to 'true' [-Wpointer-bool-conversion]
  if (!inhibit_window_system && !will_dump)
                                ~^~~~~~~~~
dispnew.c:6069:34: note: prefix with the address-of operator to silence this
      warning
  if (!inhibit_window_system && !will_dump)
                                 ^
                                 &
dispnew.c:6069:34: note: suffix with parentheses to turn this into a function
      call
  if (!inhibit_window_system && !will_dump)
                                 ^
                                          ()
dispnew.c:6176:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("internal-show-cursor", Finternal_show_cursor,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
dispnew.c:6192:1: error: argument to 'section' attribute is not valid for this
      target: mach-o section specifier requires a segment and section separated
      by a comma
DEFUN ("internal-show-cursor-p", Finternal_show_cursor_p,
^
./lisp.h:3183:4: note: expanded from macro 'DEFUN'
   SUBR_SECTION_ATTRIBUTE                                               \
   ^
./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
# define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
                                                   ^
1 warning and 11 errors generated.
-- 
Alan Third

Re: Preview: portable dumper

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1609 bytes --]

Alan Third <alan@idiocy.org> schrieb am Mi., 14. Feb. 2018 um 21:35 Uhr:

> On Wed, Feb 14, 2018 at 06:29:54AM +0200, Eli Zaretskii wrote:
> > > Date: Tue, 13 Feb 2018 23:06:15 +0100 (CET)
> > > From: Angelo Graziosi <angelo.g0@libero.it>
> > >
> > > Eli Zaretskii wrote:
> > > >
> > > > I'd urge people to try this branch and report any issues they see.
> > >
> > > What should we look for?
> >
> > The usual stuff, I think: does it build, does it work as expected,
> > etc.
>
> Lots of errors on macOS 10.13 (clang). Looks like they’re mostly the
> same thing repeated.
>
> dispnew.c:3019:1: error: argument to 'section' attribute is not valid for
> this
>       target: mach-o section specifier requires a segment and section
> separated
>       by a comma
> DEFUN ("redraw-frame", Fredraw_frame, Sredraw_frame, 0, 1, 0,
> ^
> ./lisp.h:3183:4: note: expanded from macro 'DEFUN'
>    SUBR_SECTION_ATTRIBUTE                                               \
>    ^
> ./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
> # define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
>                                                    ^
> dispnew.c:3028:1: error: argument to 'section' attribute is not valid for
> this
>       target: mach-o section specifier requires a segment and section
> separated
>       by a comma
>

The segment should probably be __DATA, cf.
https://developer.apple.com/library/content/documentation/DeveloperTools/Reference/Assembler/040-Assembler_Directives/asm_directives.html#//apple_ref/doc/uid/TP30000823-TPXREF136
.

[-- Attachment #2: Type: text/html, Size: 2357 bytes --]

Re: Preview: portable dumper

[Daniel Colascione]

On 02/14/2018 12:34 PM, Alan Third wrote:
> On Wed, Feb 14, 2018 at 06:29:54AM +0200, Eli Zaretskii wrote:
>>> Date: Tue, 13 Feb 2018 23:06:15 +0100 (CET)
>>> From: Angelo Graziosi <angelo.g0@libero.it>
>>>
>>> Eli Zaretskii wrote:
>>>>
>>>> I'd urge people to try this branch and report any issues they see.
>>>
>>> What should we look for?
>>
>> The usual stuff, I think: does it build, does it work as expected,
>> etc.
> 
> Lots of errors on macOS 10.13 (clang). Looks like they’re mostly the
> same thing repeated.
> 
> dispnew.c:3019:1: error: argument to 'section' attribute is not valid for this
>        target: mach-o section specifier requires a segment and section separated
>        by a comma
> DEFUN ("redraw-frame", Fredraw_frame, Sredraw_frame, 0, 1, 0,
> ^
> ./lisp.h:3183:4: note: expanded from macro 'DEFUN'
>     SUBR_SECTION_ATTRIBUTE                                               \
>     ^
> ./lisp.h:3150:52: note: expanded from macro 'SUBR_SECTION_ATTRIBUTE'
> # define SUBR_SECTION_ATTRIBUTE ATTRIBUTE_SECTION (".subrs")
>                                                     ^
> dispnew.c:3028:1: error: argument to 'section' attribute is not valid for this
>        target: mach-o section specifier requires a segment and section separated
>        by a comma
> DEFUN ("redraw-display", Fredraw_display, Sredraw_display, 0, 0, "",
> ^

Thanks. I've pushed a fix, I think.

Re: Preview: portable dumper

[Alan Third]

On Wed, Feb 14, 2018 at 04:49:43PM -0800, Daniel Colascione wrote:
> Thanks. I've pushed a fix, I think.

Got some more. I had to make this change:

modified   src/dispnew.c
@@ -6066,7 +6066,7 @@ init_display (void)
 #endif /* HAVE_NTGUI */
 
 #ifdef HAVE_NS
-  if (!inhibit_window_system && !will_dump)
+  if (!inhibit_window_system && !will_dump_p ())
     {
       Vinitial_window_system = Qns;
       Vwindow_system_version = make_number (10);

and now I get these:

pdumper.c:2904:21: error: implicit conversion turns string literal into bool:
      'char [48]' to 'bool' [-Werror,-Wstring-conversion]
          eassert (!"should not be dumping int: is self-representing");
                   ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./lisp.h:195:43: note: expanded from macro 'eassert'
# define eassert(cond) ((void) (false && (cond))) /* Check COND compiles.  */
                                          ^~~~
pdumper.c:3217:25: error: implicit conversion loses integer precision:
      'EMACS_INT' (aka 'long') to 'enum cold_op' [-Werror,-Wshorten-64-to-32]
      enum cold_op op = XFASTINT (XCAR (item));
                   ~~   ^~~~~~~~~~~~~~~~~~~~~~
pdumper.c:3390:31: error: implicit conversion loses integer precision:
      'EMACS_INT' (aka 'long') to 'enum dump_fixup_type'
      [-Werror,-Wshorten-64-to-32]
  enum dump_fixup_type type = XFASTINT (XCAR (fixup));
                       ~~~~   ^~~~~~~~~~~~~~~~~~~~~~~
pdumper.c:3478:32: error: implicit conversion loses integer precision:
      'EMACS_INT' (aka 'long') to 'enum dump_reloc_type'
      [-Werror,-Wshorten-64-to-32]
  dump_reloc_set_type (&reloc, XFASTINT (dump_pop (&lreloc)));
  ~~~~~~~~~~~~~~~~~~~          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pdumper.c:4726:14: error: implicit conversion from enumeration type 'const enum
      dump_reloc_type' to different enumeration type 'enum Lisp_Type'
      [-Werror,-Wenum-conversion]
    ? reloc->type
      ~~~~~~~^~~~
pdumper.c:5007:62: warning: size argument in 'memcmp' call is a comparison
      [-Wmemsize-comparison]
  if (memcmp (header->magic, dump_magic, sizeof (dump_magic) != 0))
                                         ~~~~~~~~~~~~~~~~~~~~^~~~
pdumper.c:5007:7: note: did you mean to compare the result of 'memcmp' instead?
  if (memcmp (header->magic, dump_magic, sizeof (dump_magic) != 0))
      ^                                                          ~
                                                            )
pdumper.c:5007:42: note: explicitly cast the argument to size_t to silence this
      warning
  if (memcmp (header->magic, dump_magic, sizeof (dump_magic) != 0))
                                         ^
                                         (size_t)(               )
1 warning and 5 errors generated.

I suspect some of these are because of clang.
-- 
Alan Third

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 815 bytes --]

Re: Preview: portable dumper

[Alan Third]

On Thu, Feb 15, 2018 at 11:49:30AM -0800, Daniel Colascione wrote:
> Thanks for checking. If you disable Wconversion at the top of the
> file, you should be able to unblock yourself. The warning helps make
> the code robust, but it's annoying to account for compiler
> differences.

Thanks, I’ve got it to build and it seems to work fine.

Here are the changes I had to make:

3 files changed, 15 insertions(+), 15 deletions(-)
src/dispnew.c |  2 +-
src/macfont.m | 26 +++++++++++++-------------
src/pdumper.c |  2 +-

modified   src/dispnew.c
@@ -6066,7 +6066,7 @@ init_display (void)
 #endif /* HAVE_NTGUI */
 
 #ifdef HAVE_NS
-  if (!inhibit_window_system && !will_dump)
+  if (!inhibit_window_system && !will_dump_p ())
     {
       Vinitial_window_system = Qns;
       Vwindow_system_version = make_number (10);
modified   src/macfont.m
@@ -1646,7 +1646,7 @@ static int macfont_variation_glyphs (struct font *, int c,
                                      unsigned variations[256]);
 static void macfont_filter_properties (Lisp_Object, Lisp_Object);
 
-static struct font_driver const macfont_driver =
+static struct font_driver macfont_driver =
   {
   .type = LISPSYM_INITIALLY (Qmac_ct),
   .get_cache = macfont_get_cache,
@@ -4034,6 +4034,18 @@ So we use CTFontDescriptorCreateMatchingFontDescriptor (no
 
 \f
 
+static void
+syms_of_macfont_for_pdumper (void)
+{
+  // if (RESTORING_FROM_DUMP)
+  //   macfont_family_cache = Qnil;
+  // else
+  //   eassert (NILP (macfont_family_cache));
+
+  macfont_driver.type = Qmac_ct;
+  register_font_driver (&macfont_driver, NULL);
+}
+
 void
 syms_of_macfont (void)
 {
@@ -4056,15 +4068,3 @@ So we use CTFontDescriptorCreateMatchingFontDescriptor (no
 
   pdumper_do_now_and_after_load (syms_of_macfont_for_pdumper);
 }
-
-static void
-syms_of_macfont_for_pdumper (void)
-{
-  if (RESTORING_FROM_DUMP)
-    macfont_family_cache = Qnil;
-  else
-    eassert (NILP (macfont_family_cache));
-
-  macfont_driver.type = Qmac_ct;
-  register_font_driver (&macfont_driver, NULL);
-}
modified   src/pdumper.c
@@ -55,7 +55,7 @@
 #ifdef HAVE_PDUMPER
 
 #ifdef __GNUC__
-# pragma GCC diagnostic error "-Wconversion"
+/* # pragma GCC diagnostic error "-Wconversion"  */
 # pragma GCC diagnostic error "-Wshadow"
 # define ALLOW_IMPLICIT_CONVERSION                       \
   _Pragma ("GCC diagnostic push")                        \


Presumably RESTORING_FROM_DUMP is a left‐over from previous code? I
couldn’t see at a glance what to replace it with.

Oddly whenever I had a problem with macfont.m I had to do ‘make clean’
before make would work again. I’m not sure what was going on or if
it’s specific to this branch.
-- 
Alan Third

Re: Preview: portable dumper

[Daniel Colascione]

On 02/15/2018 12:35 PM, Alan Third wrote:
> On Thu, Feb 15, 2018 at 11:49:30AM -0800, Daniel Colascione wrote:
>> Thanks for checking. If you disable Wconversion at the top of the
>> file, you should be able to unblock yourself. The warning helps make
>> the code robust, but it's annoying to account for compiler
>> differences.
> 
> Thanks, I’ve got it to build and it seems to work fine.
> 
> Here are the changes I had to make:

All fixed, I think.
> @@ -1646,7 +1646,7 @@ static int macfont_variation_glyphs (struct font *, int c,
>                                        unsigned variations[256]);
>   static void macfont_filter_properties (Lisp_Object, Lisp_Object);
>   
> -static struct font_driver const macfont_driver =
> +static struct font_driver macfont_driver =
>     {
>     .type = LISPSYM_INITIALLY (Qmac_ct),
>     .get_cache = macfont_get_cache,
> @@ -4034,6 +4034,18 @@ So we use CTFontDescriptorCreateMatchingFontDescriptor (no
>   
>   

Why did you have to remove the const?

Re: Preview: portable dumper

[Alan Third]

On Thu, Feb 15, 2018 at 02:02:03PM -0800, Daniel Colascione wrote:
> All fixed, I think.

Thanks. I’m still seeing this:

macfont.m:4057:34: error: use of undeclared identifier
      'syms_of_macfont_for_pdumper'
  pdumper_do_now_and_after_load (syms_of_macfont_for_pdumper);

I moved syms_of_macfont_for_pdumper to before syms_of_macfont to fix
it.

> > @@ -1646,7 +1646,7 @@ static int macfont_variation_glyphs (struct font *, int c,
> >                                        unsigned variations[256]);
> >   static void macfont_filter_properties (Lisp_Object, Lisp_Object);
> > -static struct font_driver const macfont_driver =
> > +static struct font_driver macfont_driver =
> >     {
> >     .type = LISPSYM_INITIALLY (Qmac_ct),
> >     .get_cache = macfont_get_cache,
> > @@ -4034,6 +4034,18 @@ So we use CTFontDescriptorCreateMatchingFontDescriptor (no
> 
> Why did you have to remove the const?

macfont.m:4045:23: error: cannot assign to variable 'macfont_driver' with
      const-qualified type 'const struct font_driver'
  macfont_driver.type = Qmac_ct;
  ~~~~~~~~~~~~~~~~~~~ ^
macfont.m:1649:33: note: variable 'macfont_driver' declared const here
static struct font_driver const macfont_driver =
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~

I think we’ll also have to copy the pdmp file into Emacs.app on
install, but I don’t know enough about how that works to do it just
now.
-- 
Alan Third

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 3438 bytes --]

Re: Preview: portable dumper

[Paul Eggert]

On 02/15/2018 03:34 PM, Daniel Colascione wrote:
> I do wonder whether it makes sense to try to copy the dump into the 
> Emacs executable itself instead of leaving it as a separate file.

I like this idea.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/15/2018 04:47 PM, Paul Eggert wrote:
> On 02/15/2018 03:34 PM, Daniel Colascione wrote:
>> I do wonder whether it makes sense to try to copy the dump into the 
>> Emacs executable itself instead of leaving it as a separate file.
> 
> I like this idea.

A simpler approach would be to just concatenate the dump to temacs, then 
add a special header with a signature and a dump length. On startup, 
Emacs could open itself, seek to the end of the file, look for the 
signature, and if it's present, back up to the start of the dump and 
proceed exactly as pdumper does now. This way, there's no overwriting 
stuff in the middle of the file and we don't waste disk space on a big 
data section we'll only partially use.

Re: Preview: portable dumper

[Eli Zaretskii]

> Cc: Eli Zaretskii <eliz@gnu.org>, Angelo Graziosi <angelo.g0@libero.it>,
>  emacs-devel@gnu.org
> From: Daniel Colascione <dancol@dancol.org>
> Date: Thu, 15 Feb 2018 17:07:29 -0800
> 
> A simpler approach would be to just concatenate the dump to temacs, then 
> add a special header with a signature and a dump length.

But it will have to be a special section, right?  Because otherwise we
again will prevent Binutils from doing useful stuff with such an
executable, right?

Re: Preview: portable dumper

[Daniel Colascione]

On 02/16/2018 12:30 AM, Eli Zaretskii wrote:
>> Cc: Eli Zaretskii <eliz@gnu.org>, Angelo Graziosi <angelo.g0@libero.it>,
>>   emacs-devel@gnu.org
>> From: Daniel Colascione <dancol@dancol.org>
>> Date: Thu, 15 Feb 2018 17:07:29 -0800
>>
>> A simpler approach would be to just concatenate the dump to temacs, then
>> add a special header with a signature and a dump length.
> 
> But it will have to be a special section, right?  Because otherwise we
> again will prevent Binutils from doing useful stuff with such an
> executable, right?

Just concatenating bytes to the end of an executable doesn't break that 
executable, at least not on any platform I know about. binutils keeps 
working. No special section required. Strip does remove the extra data 
though, unfortunately, and this procedure does break any cryptographic 
signatures (although we're not using any right now).

~/edev/trunk/src
$ readelf -h ./emacs
ELF Header:
   Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
   Class:                             ELF64
   Data:                              2's complement, little endian
   Version:                           1 (current)
   OS/ABI:                            UNIX - System V
   ABI Version:                       0
   Type:                              EXEC (Executable file)
   Machine:                           Advanced Micro Devices X86-64
   Version:                           0x1
   Entry point address:               0x41e9f0
   Start of program headers:          64 (bytes into file)
   Start of section headers:          51310168 (bytes into file)
   Flags:                             0x0
   Size of this header:               64 (bytes)
   Size of program headers:           56 (bytes)
   Number of program headers:         9
   Size of section headers:           64 (bytes)
   Number of section headers:         41
   Section header string table index: 40

~/edev/trunk/src
$ ./emacs -batch -Q --eval '(kill-emacs)'

~/edev/trunk/src
$ unzip -t kittens.zip
Archive:  kittens.zip
     testing: kittens.jpg              OK
No errors detected in compressed data of kittens.zip.

~/edev/trunk/src
$ cat emacs kittens.zip > emacs-with-kittens

~/edev/trunk/src
$ readelf -h ./emacs-with-kittens
ELF Header:
   Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
   Class:                             ELF64
   Data:                              2's complement, little endian
   Version:                           1 (current)
   OS/ABI:                            UNIX - System V
   ABI Version:                       0
   Type:                              EXEC (Executable file)
   Machine:                           Advanced Micro Devices X86-64
   Version:                           0x1
   Entry point address:               0x41e9f0
   Start of program headers:          64 (bytes into file)
   Start of section headers:          51310168 (bytes into file)
   Flags:                             0x0
   Size of this header:               64 (bytes)
   Size of program headers:           56 (bytes)
   Number of program headers:         9
   Size of section headers:           64 (bytes)
   Number of section headers:         41
   Section header string table index: 40

~/edev/trunk/src
$ chmod +x emacs-with-kittens

~/edev/trunk/src
$ ./emacs-with-kittens -batch -Q --eval '(kill-emacs)'

~/edev/trunk/src
$ unzip -t emacs-with-kittens
Archive:  emacs-with-kittens
warning [emacs-with-kittens]:  51312792 extra bytes at beginning or 
within zipfile
   (attempting to process anyway)
     testing: kittens.jpg              OK
No errors detected in compressed data of emacs-with-kittens.

Re: Preview: portable dumper

[Eli Zaretskii]

> Cc: eggert@cs.ucla.edu, alan@idiocy.org, angelo.g0@libero.it,
>  emacs-devel@gnu.org
> From: Daniel Colascione <dancol@dancol.org>
> Date: Fri, 16 Feb 2018 07:02:33 -0800
> 
> Just concatenating bytes to the end of an executable doesn't break that 
> executable, at least not on any platform I know about. binutils keeps 
> working. No special section required. Strip does remove the extra data 
> though, unfortunately

I'm guessing similar problems will happen with objcopy and objdump.

Re: Preview: portable dumper

[Andy Moreton]

On Fri 16 Feb 2018, Daniel Colascione wrote:
> Just concatenating bytes to the end of an executable doesn't break that
> executable, at least not on any platform I know about. binutils keeps working.
> No special section required. Strip does remove the extra data though,
> unfortunately, and this procedure does break any cryptographic signatures

...and that is why it should not be done. Most platforms are moving in
the direction of requiring cryptographically signed binaries.

> (although we're not using any right now).

However it will be necessary to support this in the near future.

Thanks for your work on the portable dumper though - this already looks
solid enough that the old dump code does not have long to live...

   AndyM

Re: Preview: portable dumper

[Stefan Monnier]

> I do wonder whether it makes sense to try to copy the dump into the
> Emacs executable itself instead of leaving it as a separate file.

We could try, but:
- it adds complexity and maybe system-dependent hacks.
- it closes the opportunity to have several dump files for a single
  executable (I was thinking we could try and let end-users build their
  own dump file).


        Stefan

Re: Preview: portable dumper

[Daniel Colascione]

On 02/15/2018 05:54 PM, Stefan Monnier wrote:
>> I do wonder whether it makes sense to try to copy the dump into the
>> Emacs executable itself instead of leaving it as a separate file.
> 
> We could try, but:
> - it adds complexity and maybe system-dependent hacks.

Not if we just tack onto the end.

> - it closes the opportunity to have several dump files for a single
>    executable (I was thinking we could try and let end-users build their
>    own dump file).
Not necessarily. Suppose we go with the append-to-the-executable option. 
Then, to "re-dump" emacs, we'd open the current executable, copy it to a 
temporary file, back up to the start of the dump, ftruncate. Now we've 
recovered temacs and we can go through normal loadup and dump.

Maybe we can even automatically detect changes to any file in 
load-history and perform this procedure automatically on startup.

Self-contained emacs binary?

[Daniel Colascione]

On 02/15/2018 06:25 PM, Daniel Colascione wrote:
> On 02/15/2018 05:54 PM, Stefan Monnier wrote:
>>> I do wonder whether it makes sense to try to copy the dump into the
>>> Emacs executable itself instead of leaving it as a separate file.
>>
>> We could try, but:
>> - it adds complexity and maybe system-dependent hacks.
> 
> Not if we just tack onto the end.
> 
>> - it closes the opportunity to have several dump files for a single
>>    executable (I was thinking we could try and let end-users build their
>>    own dump file).
> Not necessarily. Suppose we go with the append-to-the-executable option. 
> Then, to "re-dump" emacs, we'd open the current executable, copy it to a 
> temporary file, back up to the start of the dump, ftruncate. Now we've 
> recovered temacs and we can go through normal loadup and dump.
> 
> Maybe we can even automatically detect changes to any file in 
> load-history and perform this procedure automatically on startup.

You know, we could just append a whole zip archive to the executable and 
load elisp files from this zip archive. (The dump file would be just 
another file in the archive.) This way, we'd be able to make a 
self-contained "emacs" binary that wouldn't need special installation. 
(arc-mode would keep find-library working.) We could even mmap files 
from the zip archive as long as the files are suitably aligned and 
STOREd instead of compressed; Android uses this trick to map stuff 
sitting inside APK files.

Re: Self-contained emacs binary?

[Ken Raeburn]

On Feb 15, 2018, at 21:37, Daniel Colascione <dancol@dancol.org> wrote:

> On 02/15/2018 06:25 PM, Daniel Colascione wrote:
>> On 02/15/2018 05:54 PM, Stefan Monnier wrote:
>>>> I do wonder whether it makes sense to try to copy the dump into the
>>>> Emacs executable itself instead of leaving it as a separate file.
>>> 
>>> We could try, but:
>>> - it adds complexity and maybe system-dependent hacks.
>> Not if we just tack onto the end.
>>> - it closes the opportunity to have several dump files for a single
>>>    executable (I was thinking we could try and let end-users build their
>>>    own dump file).
>> Not necessarily. Suppose we go with the append-to-the-executable option. Then, to "re-dump" emacs, we'd open the current executable, copy it to a temporary file, back up to the start of the dump, ftruncate. Now we've recovered temacs and we can go through normal loadup and dump.
>> Maybe we can even automatically detect changes to any file in load-history and perform this procedure automatically on startup.
> 
> You know, we could just append a whole zip archive to the executable and load elisp files from this zip archive. (The dump file would be just another file in the archive.) This way, we'd be able to make a self-contained "emacs" binary that wouldn't need special installation. (arc-mode would keep find-library working.) We could even mmap files from the zip archive as long as the files are suitably aligned and STOREd instead of compressed; Android uses this trick to map stuff sitting inside APK files.

Embedding the Lisp support code (or other things) via C character arrays during the build process would also work, and would be cleaner, though it specifically fails to handle your dump-file issue, unless you do go the route of patching up an embedded array later. I do think that’s probably cleaner than appending arbitrary data to an executable.

(On a tangent to that, I’ve got some experimental patches around for embedding the C doc strings in the executable, halfway to eliminating the need for the separate DOC file.)

Tying the dump to the executable image required to make it work does seem like a win. I’m not sure I’d call it entirely “self-contained” unless you could somehow fold in the info files, support executables like movemail and hexl, etc., sort of like the “app” bundle on macOS does (really a directory but the GUI treats it as one object for moving around).

Ken

Re: Preview: portable dumper

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I do wonder whether it makes sense to try to copy the dump into the
>> Emacs executable itself instead of leaving it as a separate file.
>
> We could try, but:
> - it adds complexity and maybe system-dependent hacks.
> - it closes the opportunity to have several dump files for a single
>   executable (I was thinking we could try and let end-users build their
>   own dump file).

While I've got little to add the discussion of the innards, this
occurred to me also. In fact, I've been trying it -- I figured that
a "redump Emacs after load .emacs" would be a fairly attractive option,
especially if it was automated.

Anyway, at the moment, I can't get it to work. Calling this:

(dump-emacs-portable "~/scratch/test.pdmp")

after loading my .emacs works (early versions crashed, so something's
been fixed there). However, trying to use this:

./src/emacs -q --dump-file ~/scratch/test.pdmp 
emacs: could not load dump file "/home/phillord/scratch/test.pdmp": not a dump file

Phil

Re: Preview: portable dumper

[Daniel Colascione]

On 02/21/2018 02:04 PM, Phillip Lord wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
>>> I do wonder whether it makes sense to try to copy the dump into the
>>> Emacs executable itself instead of leaving it as a separate file.
>>
>> We could try, but:
>> - it adds complexity and maybe system-dependent hacks.
>> - it closes the opportunity to have several dump files for a single
>>    executable (I was thinking we could try and let end-users build their
>>    own dump file).
> 
> While I've got little to add the discussion of the innards, this
> occurred to me also. In fact, I've been trying it -- I figured that
> a "redump Emacs after load .emacs" would be a fairly attractive option,
> especially if it was automated.
> 
> Anyway, at the moment, I can't get it to work. Calling this:
> 
> (dump-emacs-portable "~/scratch/test.pdmp")
> 
> after loading my .emacs works (early versions crashed, so something's
> been fixed there). However, trying to use this:
> 
> ./src/emacs -q --dump-file ~/scratch/test.pdmp
> emacs: could not load dump file "/home/phillord/scratch/test.pdmp": not a dump file

I'm able to re-dump now. Can you try it again?

Re: Preview: portable dumper

[Phillip Lord]

Daniel Colascione <dancol@dancol.org> writes:

> On 02/21/2018 02:04 PM, Phillip Lord wrote:
>> Anyway, at the moment, I can't get it to work. Calling this:
>>
>> (dump-emacs-portable "~/scratch/test.pdmp")
>>
>> after loading my .emacs works (early versions crashed, so something's
>> been fixed there). However, trying to use this:
>>
>> ./src/emacs -q --dump-file ~/scratch/test.pdmp
>> emacs: could not load dump file "/home/phillord/scratch/test.pdmp": not a dump file
>
> I'm able to re-dump now. Can you try it again?

Dumping from ./src/emacs -q

gives me this in *Messages* after the dump (although the call to
dump-emacs-portable returns without error).

eval: unsupported object type in dump: SAVE_POINTER

and this on trying to reload with

./src/emacs -q --dump-file ~/scratch/test.pdmp

Gives:

dump file is result of failed dump attempt

on the command line. On an emacs started with

./src/emacs

and running the various stuff in my .emacs, I get this:

eval: unsupported object type in dump: window configuration

Likewise, cannot restart with --dump-file. This is running on Ubuntu
16.04.3, 64 bit.

Phil

Re: Preview: portable dumper

[Clément Pit-Claudel]

On 2018-02-26 10:10, Phillip Lord wrote:
> eval: unsupported object type in dump: SAVE_POINTER

You can pass t to dump-emacs-portable to see where the unsupported values come from.  For me SAVE_POINTERs came from loading dbus, and window configurations from loading winner-mode.

Clément.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/26/2018 07:23 AM, Clément Pit-Claudel wrote:
> On 2018-02-26 10:10, Phillip Lord wrote:
>> eval: unsupported object type in dump: SAVE_POINTER
> 
> You can pass t to dump-emacs-portable to see where the unsupported values come from.  For me SAVE_POINTERs came from loading dbus, and window configurations from loading winner-mode.

I love when people use features that aren't even properly documented 
yet. :-)

I'm imagining using 'with-wrapper-hook' (or, since it's deprecated, 
'add-function') to give modes a chance to tweak the global lisp 
environment that pdumper sees, avoiding some of these problems. Another 
option, which isn't necessarily mutually exclusive, is to provide a list 
of symbols to dump as nil.

For now, you're going to get the most reliable results dumping a -batch 
Emacs.

Re: Preview: portable dumper

[Andy Moreton]

On Mon 26 Feb 2018, Daniel Colascione wrote:

> On 02/26/2018 07:23 AM, Clément Pit-Claudel wrote:
>> On 2018-02-26 10:10, Phillip Lord wrote:
>>> eval: unsupported object type in dump: SAVE_POINTER
>>
>> You can pass t to dump-emacs-portable to see where the unsupported values
>> come from. For me SAVE_POINTERs came from loading dbus, and window
>> configurations from loading winner-mode.
>
> I love when people use features that aren't even properly documented yet. :-)

For the benefit of readers of this list who have been asked to test the
pdumper branch and report their findings, can you please describe what is
expected to work correctly, and which areas still need additional work.

Please add the "--dump-file" command line argument to the command line
help in usage_message[] in emacs.c.

    AndyM

Re: Preview: portable dumper

[Phillip Lord]

Daniel Colascione <dancol@dancol.org> writes:

> On 02/26/2018 07:23 AM, Clément Pit-Claudel wrote:
>> On 2018-02-26 10:10, Phillip Lord wrote:
>>> eval: unsupported object type in dump: SAVE_POINTER
>>
>> You can pass t to dump-emacs-portable to see where the unsupported
>> values come from.  For me SAVE_POINTERs came from loading dbus, and
>> window configurations from loading winner-mode.
>
> I love when people use features that aren't even properly documented
> yet. :-)
>
> I'm imagining using 'with-wrapper-hook' (or, since it's deprecated,
> 'add-function') to give modes a chance to tweak the global lisp
> environment that pdumper sees, avoiding some of these
> problems. Another option, which isn't necessarily mutually exclusive,
> is to provide a list of symbols to dump as nil.
>
> For now, you're going to get the most reliable results dumping a
> -batch Emacs.

I assumed that this was the case. Do you want bug reports about dumping
a non-batch? I think the use case of speeding up start up is a good one
to address, but I am aware that it's an add-on that Emacs doesn't
support at the moment. Getting the dumper stable for normal development
use is the main thing.

Phil

Re: Preview: portable dumper

[Eli Zaretskii]

> Date: Thu, 15 Feb 2018 15:34:13 -0800
> From: Daniel Colascione <dancol@dancol.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, Angelo Graziosi <angelo.g0@libero.it>,
>  emacs-devel@gnu.org
> 
> I do wonder whether it makes sense to try to copy the dump into the Emacs executable itself instead of
> leaving it as a separate file. We could do it independently of executable format by defining a data array in static
> storage that's initially full of, say, 15MB of zeroes prefixed by a long random header (like a MIME boundary),
> then, after we generate emacs.pdmp, copying the dump file into the executable at the place where we see
> that random header. If the dump turns out to be bigger than that 15MB, we can fail the build and ask the user
> to enlarge the array.
> 
> I don't know of any executable format for which this scheme would fail.

Wouldn't that make the dumper stuff less portable, in the sense that
it would need to be compatible with low-level details of executable
file formats on various systems?

At least on non-ELF systems, AFAIK the flexibility of putting
arbitrary sections into an executable is lower than desired.  For
example, before Emacs 25 the MS-Windows build would create a special
section for the initialized Emacs data, which had the annoying effect
of running afoul of 'strip', because Binutils don't know about this
section, and therefore stripping would produce a dysfunctional
executable.  It also prevented re-dumping Emacs, something we had in
the past and I'd like us to have again in the future.

Wouldn't copying the dump into the executable hit the same problems,
at least in principle?

> This way, we could get rid of the current complicated dump-file location logic early in Emacs init.

Can you elaborate why is that logic so complicated?

Re: Preview: portable dumper

[Andy Moreton]

On Fri 16 Feb 2018, Eli Zaretskii wrote:

>> Date: Thu, 15 Feb 2018 15:34:13 -0800
>> From: Daniel Colascione <dancol@dancol.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, Angelo Graziosi <angelo.g0@libero.it>,
>>  emacs-devel@gnu.org
>> 
>> I do wonder whether it makes sense to try to copy the dump into the Emacs executable itself instead of
>> leaving it as a separate file. We could do it independently of executable format by defining a data array in static
>> storage that's initially full of, say, 15MB of zeroes prefixed by a long random header (like a MIME boundary),
>> then, after we generate emacs.pdmp, copying the dump file into the executable at the place where we see
>> that random header. If the dump turns out to be bigger than that 15MB, we can fail the build and ask the user
>> to enlarge the array.
>> 
>> I don't know of any executable format for which this scheme would fail.
>
> Wouldn't that make the dumper stuff less portable, in the sense that
> it would need to be compatible with low-level details of executable
> file formats on various systems?
>
> At least on non-ELF systems, AFAIK the flexibility of putting
> arbitrary sections into an executable is lower than desired.  For
> example, before Emacs 25 the MS-Windows build would create a special
> section for the initialized Emacs data, which had the annoying effect
> of running afoul of 'strip', because Binutils don't know about this
> section, and therefore stripping would produce a dysfunctional
> executable.  It also prevented re-dumping Emacs, something we had in
> the past and I'd like us to have again in the future.
>
> Wouldn't copying the dump into the executable hit the same problems,
> at least in principle?

I don't understand the desire to put the dump within the eamcs
executable, as I thought the whole point of this exercise was to avoid
dodgy manipulation of executable file formats.

Anything that fiddles with the executable prevents signing the
executable, which in many environments is desireable.

    AndyM

Re: Preview: portable dumper

[Daniel Colascione]

On 02/16/2018 03:30 AM, Andy Moreton wrote:
> On Fri 16 Feb 2018, Eli Zaretskii wrote:
> 
>>> Date: Thu, 15 Feb 2018 15:34:13 -0800
>>> From: Daniel Colascione <dancol@dancol.org>
>>> Cc: Eli Zaretskii <eliz@gnu.org>, Angelo Graziosi <angelo.g0@libero.it>,
>>>   emacs-devel@gnu.org
>>>
>>> I do wonder whether it makes sense to try to copy the dump into the Emacs executable itself instead of
>>> leaving it as a separate file. We could do it independently of executable format by defining a data array in static
>>> storage that's initially full of, say, 15MB of zeroes prefixed by a long random header (like a MIME boundary),
>>> then, after we generate emacs.pdmp, copying the dump file into the executable at the place where we see
>>> that random header. If the dump turns out to be bigger than that 15MB, we can fail the build and ask the user
>>> to enlarge the array.
>>>
>>> I don't know of any executable format for which this scheme would fail.
>>
>> Wouldn't that make the dumper stuff less portable, in the sense that
>> it would need to be compatible with low-level details of executable
>> file formats on various systems?
>>
>> At least on non-ELF systems, AFAIK the flexibility of putting
>> arbitrary sections into an executable is lower than desired.  For
>> example, before Emacs 25 the MS-Windows build would create a special
>> section for the initialized Emacs data, which had the annoying effect
>> of running afoul of 'strip', because Binutils don't know about this
>> section, and therefore stripping would produce a dysfunctional
>> executable.  It also prevented re-dumping Emacs, something we had in
>> the past and I'd like us to have again in the future.
>>
>> Wouldn't copying the dump into the executable hit the same problems,
>> at least in principle?
> 
> I don't understand the desire to put the dump within the eamcs
> executable, as I thought the whole point of this exercise was to avoid
> dodgy manipulation of executable file formats.

It's just a matter of packaging. As I mentioned in the other message I 
sent, keeping the dump separate from the executable breaks some 
perfectly-natural things you might want to do with the Emacs executable, 
like copying and renaming it. If possible, I'd like to keep these things 
working. If we can't, that's a shame, but it's not the end of the world.

We do need to avoid platform-specific executable file manipulation and 
weird runtime hacks. There are three options on the table:

1) Keeping the dump as a separate file,
2) Appending the dump to the end of the executable, and
3) Overwriting a data-section array with the dump contents.

All of these options avoid dodgy platform-specific executable 
modifications of the sort you see in, say, unexw32.c, and they're all 
compatible with emacs as a position-independent executable.

> Anything that fiddles with the executable prevents signing the
> executable, which in many environments is desireable.

Ideally, we'd be able to re-sign after modification.

Re: Preview: portable dumper

[Robert Pluim]

Daniel Colascione <dancol@dancol.org> writes:

>
> We do need to avoid platform-specific executable file manipulation and
> weird runtime hacks. There are three options on the table:
>
> 1) Keeping the dump as a separate file,
> 2) Appending the dump to the end of the executable, and
> 3) Overwriting a data-section array with the dump contents.

If I remember correctly, XEmacs chose option (1), but that was a long
time ago. If we implement that, nothing prevents us from adding option
(2) later. Later builds that implement (2) would then simply not
provide the dump file separately, and once it's shown to work reliably
the code for (1) could be removed (or (1) and (2) could be swapped).

Option 3 seems less flexible, plus as someone mentioned it would be
nice to able to redump your own emacs, which could run into a
limitation of the size of the data section.

Robert

Re: Preview: portable dumper

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 2241 bytes --]

Andy Moreton <andrewjmoreton@gmail.com> schrieb am Fr., 16. Feb. 2018 um
12:31 Uhr:

> On Fri 16 Feb 2018, Eli Zaretskii wrote:
>
> >> Date: Thu, 15 Feb 2018 15:34:13 -0800
> >> From: Daniel Colascione <dancol@dancol.org>
> >> Cc: Eli Zaretskii <eliz@gnu.org>, Angelo Graziosi <angelo.g0@libero.it
> >,
> >>  emacs-devel@gnu.org
> >>
> >> I do wonder whether it makes sense to try to copy the dump into the
> Emacs executable itself instead of
> >> leaving it as a separate file. We could do it independently of
> executable format by defining a data array in static
> >> storage that's initially full of, say, 15MB of zeroes prefixed by a
> long random header (like a MIME boundary),
> >> then, after we generate emacs.pdmp, copying the dump file into the
> executable at the place where we see
> >> that random header. If the dump turns out to be bigger than that 15MB,
> we can fail the build and ask the user
> >> to enlarge the array.
> >>
> >> I don't know of any executable format for which this scheme would fail.
> >
> > Wouldn't that make the dumper stuff less portable, in the sense that
> > it would need to be compatible with low-level details of executable
> > file formats on various systems?
> >
> > At least on non-ELF systems, AFAIK the flexibility of putting
> > arbitrary sections into an executable is lower than desired.  For
> > example, before Emacs 25 the MS-Windows build would create a special
> > section for the initialized Emacs data, which had the annoying effect
> > of running afoul of 'strip', because Binutils don't know about this
> > section, and therefore stripping would produce a dysfunctional
> > executable.  It also prevented re-dumping Emacs, something we had in
> > the past and I'd like us to have again in the future.
> >
> > Wouldn't copying the dump into the executable hit the same problems,
> > at least in principle?
>
> I don't understand the desire to put the dump within the eamcs
> executable, as I thought the whole point of this exercise was to avoid
> dodgy manipulation of executable file formats.
>

I agree. We already ship a lot of files that are expected at certain
locations, like the Emacs Lisp files or the files in `data-directory', why
should the pdump file be different?

[-- Attachment #2: Type: text/html, Size: 3018 bytes --]

Re: Preview: portable dumper

[Daniel Colascione]

On 02/16/2018 09:00 AM, Philipp Stephani wrote:
> Andy Moreton <andrewjmoreton@gmail.com 
> <mailto:andrewjmoreton@gmail.com>> schrieb am Fr., 16. Feb. 2018 um 
> 12:31 Uhr:
> 
>     On Fri 16 Feb 2018, Eli Zaretskii wrote:
> 
>      >> Date: Thu, 15 Feb 2018 15:34:13 -0800
>      >> From: Daniel Colascione <dancol@dancol.org
>     <mailto:dancol@dancol.org>>
>      >> Cc: Eli Zaretskii <eliz@gnu.org <mailto:eliz@gnu.org>>, Angelo
>     Graziosi <angelo.g0@libero.it <mailto:angelo.g0@libero.it>>,
>      >> emacs-devel@gnu.org <mailto:emacs-devel@gnu.org>
>      >>
>      >> I do wonder whether it makes sense to try to copy the dump into
>     the Emacs executable itself instead of
>      >> leaving it as a separate file. We could do it independently of
>     executable format by defining a data array in static
>      >> storage that's initially full of, say, 15MB of zeroes prefixed
>     by a long random header (like a MIME boundary),
>      >> then, after we generate emacs.pdmp, copying the dump file into
>     the executable at the place where we see
>      >> that random header. If the dump turns out to be bigger than that
>     15MB, we can fail the build and ask the user
>      >> to enlarge the array.
>      >>
>      >> I don't know of any executable format for which this scheme
>     would fail.
>      >
>      > Wouldn't that make the dumper stuff less portable, in the sense that
>      > it would need to be compatible with low-level details of executable
>      > file formats on various systems?
>      >
>      > At least on non-ELF systems, AFAIK the flexibility of putting
>      > arbitrary sections into an executable is lower than desired.  For
>      > example, before Emacs 25 the MS-Windows build would create a special
>      > section for the initialized Emacs data, which had the annoying effect
>      > of running afoul of 'strip', because Binutils don't know about this
>      > section, and therefore stripping would produce a dysfunctional
>      > executable.  It also prevented re-dumping Emacs, something we had in
>      > the past and I'd like us to have again in the future.
>      >
>      > Wouldn't copying the dump into the executable hit the same problems,
>      > at least in principle?
> 
>     I don't understand the desire to put the dump within the eamcs
>     executable, as I thought the whole point of this exercise was to avoid
>     dodgy manipulation of executable file formats.
> 
> 
> I agree. We already ship a lot of files that are expected at certain 
> locations, like the Emacs Lisp files or the files in `data-directory', 
> why should the pdump file be different?

The files you've mentioned aren't as intimately tied to the Emacs binary 
as the pdmp file is. You can load these elc and data files into any old 
Emacs, but the dump has to be an _exact_ match.

Re: Preview: portable dumper

[Daniel Colascione]

On 02/16/2018 12:24 AM, Eli Zaretskii wrote:
>> Date: Thu, 15 Feb 2018 15:34:13 -0800
>> From: Daniel Colascione <dancol@dancol.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, Angelo Graziosi <angelo.g0@libero.it>,
>>   emacs-devel@gnu.org
>>
>> I do wonder whether it makes sense to try to copy the dump into the Emacs executable itself instead of
>> leaving it as a separate file. We could do it independently of executable format by defining a data array in static
>> storage that's initially full of, say, 15MB of zeroes prefixed by a long random header (like a MIME boundary),
>> then, after we generate emacs.pdmp, copying the dump file into the executable at the place where we see
>> that random header. If the dump turns out to be bigger than that 15MB, we can fail the build and ask the user
>> to enlarge the array.
>>
>> I don't know of any executable format for which this scheme would fail.
> 
> Wouldn't that make the dumper stuff less portable, in the sense that
> it would need to be compatible with low-level details of executable
> file formats on various systems?

No: I'm thinking about only options that work independently of 
particular executable formats.

> At least on non-ELF systems, AFAIK the flexibility of putting
> arbitrary sections into an executable is lower than desired.  For
> example, before Emacs 25 the MS-Windows build would create a special
> section for the initialized Emacs data, which had the annoying effect
> of running afoul of 'strip', because Binutils don't know about this
> section, and therefore stripping would produce a dysfunctional
> executable.  

That does sound annoying. The original proposal I had in mind was to 
make a normal data array:

uint8_t dump[15*1024*1024] = { embedded_dump_signature };

And then refer to this array on code. No part of the toolchain is 
allowed to remove this array. The "dump insertion" procedure we're 
considering would just amount to changing the contents of this array in 
an already-linked executable using dumb find-and-replace anchored on 
embedded_dump_signature.

> It also prevented re-dumping Emacs, something we had in
> the past and I'd like us to have again in the future.

I'd like to have it too. The data-section-array approach above wouldn't 
permit redumping unless the new dump fit into the old array. The 
concatenation approach would allow arbitrary redumping. And of course 
keeping the dump as a separate file would too.

> Wouldn't copying the dump into the executable hit the same problems,
> at least in principle?
> 
>> This way, we could get rid of the current complicated dump-file location logic early in Emacs init.
> 
> Can you elaborate why is that logic so complicated?

Two reasons:

1) The code to generate the dump file is complicated, especially because 
we want to support windows, and
2) with a separate dump file, it's easier to accidentally break the 
correspondence between the dump and the emacs binary itself. If we 
somehow stapled the dump to the binary, it'd be a lot harder to break 
this way.

I like the idea of being able to mv emacs emacs.old, make emacs, and 
then have emacs.old and emacs still both run and work normally. I don't 
think the numbered build mechanism you fixed completely solves the 
problem, since it only works in one specific context.

Re: Preview: portable dumper

[Eli Zaretskii]

> Cc: alan@idiocy.org, angelo.g0@libero.it, emacs-devel@gnu.org
> From: Daniel Colascione <dancol@dancol.org>
> Date: Fri, 16 Feb 2018 07:10:42 -0800
> 
> The original proposal I had in mind was to make a normal data array:
> 
> uint8_t dump[15*1024*1024] = { embedded_dump_signature };
> 
> And then refer to this array on code. No part of the toolchain is 
> allowed to remove this array. The "dump insertion" procedure we're 
> considering would just amount to changing the contents of this array in 
> an already-linked executable using dumb find-and-replace anchored on 
> embedded_dump_signature.

I can understand how rewriting the array could work (but we'd need to
know its size in advance, otherwise we'd waste memory, like Emacs on
w32 does today with its dumped_data array, especially since the size
during bootstrap needs to be so much larger than with byte-compiled
Lisp files).  But I'm not sure I understand how references to that
array in the code would work.  Do you mean there would be a need to
"fix up" address references in the code as part of "dump insertion"?
If so, that's my original suggestion from 2 years ago, which was voted
down due to complexities.

> I like the idea of being able to mv emacs emacs.old, make emacs, and 
> then have emacs.old and emacs still both run and work normally. I don't 
> think the numbered build mechanism you fixed completely solves the 
> problem, since it only works in one specific context.

I imagined the fingerprint we compute is already used (or can be used)
to reject dump files from a different build?  If so, this could
provide at least some protection form calamities.

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 3765 bytes --]

Re: Preview: portable dumper

[Eli Zaretskii]

> Date: Fri, 16 Feb 2018 07:44:50 -0800
> From: Daniel Colascione <dancol@dancol.org>
> Cc: alan@idiocy.org, angelo.g0@libero.it, emacs-devel@gnu.org
> 
> That the dump-array proposal caps
> the size of the dump (which will probably break user redumping) and that strip(1) breaks the concatenation
> approach suggests that maybe the existing separate file option is right after all.

I do indeed think that we should make the separate file method stable
and collect some experience with it, before consider more advanced
and/or clever approaches.

Re: Preview: portable dumper

[Stefan Monnier]

> That the dump-array proposal caps the size of the dump (which will
> probably break user redumping) and that strip(1) breaks the
> concatenation approach suggests that maybe the existing separate file
> option is right after all.

IIUC these options actually aren't mutually exclusive.
We can easily provide a single `(t)emacs` executable which at startup
will:
- look for a valid dump in the big static `dump` array.
- if not found look for a valid dump in a particular ELF section.
- if not found, look for a valid dump at the end of the `emacs` binary.
- if not found, look for a dump as a separate file at some standard
  location (starting with ~/.emacs.d/<magicid>.dump, say).
- as support a "--preload <foo>.dump" command-line option.


        Stefan

Re: Preview: portable dumper

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

Alan Third <alan@idiocy.org> schrieb am Do., 15. Feb. 2018 um 21:36 Uhr:

> On Thu, Feb 15, 2018 at 11:49:30AM -0800, Daniel Colascione wrote:
> > Thanks for checking. If you disable Wconversion at the top of the
> > file, you should be able to unblock yourself. The warning helps make
> > the code robust, but it's annoying to account for compiler
> > differences.
>
> Thanks, I’ve got it to build and it seems to work fine.
>
> Here are the changes I had to make:
>
> 3 files changed, 15 insertions(+), 15 deletions(-)
> src/dispnew.c |  2 +-
> src/macfont.m | 26 +++++++++++++-------------
> src/pdumper.c |  2 +-
>

Another minor thing (that breaks the build with -Werror) is that the
functions set_{marker,vectorlike}_marked are unused. Same with
syms_of_nsfns_1. I also had to change the eassert(("...", 0)) to
eassert("..." && 0) to stop the compiler from complaining about an unused
expression.

[-- Attachment #2: Type: text/html, Size: 1264 bytes --]

Re: Preview: portable dumper

[Philipp Stephani]

[-- Attachment #1.1: Type: text/plain, Size: 1089 bytes --]

Philipp Stephani <p.stephani2@gmail.com> schrieb am Do., 15. Feb. 2018 um
23:38 Uhr:

> Alan Third <alan@idiocy.org> schrieb am Do., 15. Feb. 2018 um 21:36 Uhr:
>
>> On Thu, Feb 15, 2018 at 11:49:30AM -0800, Daniel Colascione wrote:
>> > Thanks for checking. If you disable Wconversion at the top of the
>> > file, you should be able to unblock yourself. The warning helps make
>> > the code robust, but it's annoying to account for compiler
>> > differences.
>>
>> Thanks, I’ve got it to build and it seems to work fine.
>>
>> Here are the changes I had to make:
>>
>> 3 files changed, 15 insertions(+), 15 deletions(-)
>> src/dispnew.c |  2 +-
>> src/macfont.m | 26 +++++++++++++-------------
>> src/pdumper.c |  2 +-
>>
>
> Another minor thing (that breaks the build with -Werror) is that the
> functions set_{marker,vectorlike}_marked are unused. Same with
> syms_of_nsfns_1. I also had to change the eassert(("...", 0)) to
> eassert("..." && 0) to stop the compiler from complaining about an unused
> expression.
>

This is the patch that I've applied:

[-- Attachment #1.2: Type: text/html, Size: 1726 bytes --]

[-- Attachment #2: pdump-patch.txt --]
[-- Type: text/plain, Size: 3182 bytes --]

diff --git a/src/alloc.c b/src/alloc.c
index 90f913da36..6f81195cc3 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -378,7 +378,6 @@ inline static bool vector_marked_p (const struct Lisp_Vector *v);
 inline static void set_vector_marked (struct Lisp_Vector *v);
 
 inline static bool vectorlike_marked_p (const union vectorlike_header *v);
-inline static void set_vectorlike_marked (union vectorlike_header *v);
 
 inline static bool cons_marked_p (const struct Lisp_Cons *c);
 inline static void set_cons_marked (struct Lisp_Cons *c);
@@ -393,7 +392,6 @@ inline static bool misc_any_marked_p (const struct Lisp_Misc_Any *m);
 inline static void set_misc_any_marked (struct Lisp_Misc_Any *m);
 
 inline static bool marker_marked_p (const struct Lisp_Marker *m);
-inline static void set_marker_marked (struct Lisp_Marker *m);
 
 inline static bool overlay_marked_p (const struct Lisp_Overlay *m);
 inline static void set_overlay_marked (struct Lisp_Overlay *m);
@@ -4150,12 +4148,6 @@ vectorlike_marked_p (const union vectorlike_header *header)
   return vector_marked_p ((const struct Lisp_Vector *) header);
 }
 
-static void
-set_vectorlike_marked (union vectorlike_header *header)
-{
-  set_vector_marked ((struct Lisp_Vector *) header);
-}
-
 static bool
 cons_marked_p (const struct Lisp_Cons *c)
 {
@@ -4230,12 +4222,6 @@ marker_marked_p (const struct Lisp_Marker *m)
   return misc_any_marked_p ((struct Lisp_Misc_Any *) m);
 }
 
-static void
-set_marker_marked (struct Lisp_Marker *m)
-{
-  set_misc_any_marked ((struct Lisp_Misc_Any *) m);
-}
-
 static bool
 overlay_marked_p (const struct Lisp_Overlay *m)
 {
diff --git a/src/macfont.m b/src/macfont.m
index 377fc3fa48..2cbe07d751 100644
--- a/src/macfont.m
+++ b/src/macfont.m
@@ -1646,7 +1646,7 @@ static int macfont_variation_glyphs (struct font *, int c,
                                      unsigned variations[256]);
 static void macfont_filter_properties (Lisp_Object, Lisp_Object);
 
-static struct font_driver const macfont_driver =
+static struct font_driver macfont_driver =
   {
   .type = LISPSYM_INITIALLY (Qmac_ct),
   .get_cache = macfont_get_cache,
@@ -4034,6 +4034,8 @@ So we use CTFontDescriptorCreateMatchingFontDescriptor (no
 
 \f
 
+static void syms_of_macfont_for_pdumper (void);
+
 void
 syms_of_macfont (void)
 {
diff --git a/src/nsfns.m b/src/nsfns.m
index 8911ad27fe..c2ee4df33f 100644
--- a/src/nsfns.m
+++ b/src/nsfns.m
@@ -3196,12 +3196,6 @@ - (NSString *)panel: (id)sender userEnteredFilename: (NSString *)filename
 
    ========================================================================== */
 
-static void
-syms_of_nsfns_1 (void)
-{
-
-}
-
 void
 syms_of_nsfns (void)
 {
diff --git a/src/pdumper.c b/src/pdumper.c
index 499c8dc27c..7053e52ad6 100644
--- a/src/pdumper.c
+++ b/src/pdumper.c
@@ -2917,7 +2917,7 @@ dump_object_1 (struct dump_context *ctx, Lisp_Object object)
           offset = dump_float (ctx, XFLOAT (object));
           break;
         case_Lisp_Int:
-          eassert (("should not be dumping int: is self-representing", 0));
+          eassert ("should not be dumping int: is self-representing" && false);
         default:
           emacs_abort ();
         }

Re: Preview: portable dumper

[Yoshiaki Kasahara]

Hello,

I cannot build pdumper branch on FreeBSD 11.1-STABLE with gcc 6.4.0.
I guess INLINE macro already contains 'static' on FreeBSD.

  CC       alloc.o
alloc.c:377:1: error: duplicate 'static'
 INLINE static bool vector_marked_p (const struct Lisp_Vector *v);
 ^~~~~~
alloc.c:378:1: error: duplicate 'static'
 INLINE static void set_vector_marked (struct Lisp_Vector *v);
 ^~~~~~
alloc.c:380:1: error: duplicate 'static'
 INLINE static bool vectorlike_marked_p (const union vectorlike_header *v);
 ^~~~~~
alloc.c:381:1: error: duplicate 'static'
 INLINE static void set_vectorlike_marked (union vectorlike_header *v);
 ^~~~~~
alloc.c:383:1: error: duplicate 'static'
 INLINE static bool cons_marked_p (const struct Lisp_Cons *c);
 ^~~~~~
alloc.c:384:1: error: duplicate 'static'
 INLINE static void set_cons_marked (struct Lisp_Cons *c);
 ^~~~~~
alloc.c:386:1: error: duplicate 'static'
 INLINE static bool string_marked_p (const struct Lisp_String *s);
 ^~~~~~
alloc.c:387:1: error: duplicate 'static'
 INLINE static void set_string_marked (struct Lisp_String *s);
 ^~~~~~
alloc.c:389:1: error: duplicate 'static'
 INLINE static bool symbol_marked_p (const struct Lisp_Symbol *s);
 ^~~~~~
alloc.c:390:1: error: duplicate 'static'
 INLINE static void set_symbol_marked (struct Lisp_Symbol *s);
 ^~~~~~
alloc.c:392:1: error: duplicate 'static'
 INLINE static bool misc_any_marked_p (const struct Lisp_Misc_Any *m);
 ^~~~~~
alloc.c:393:1: error: duplicate 'static'
 INLINE static void set_misc_any_marked (struct Lisp_Misc_Any *m);
 ^~~~~~
alloc.c:395:1: error: duplicate 'static'
 INLINE static bool marker_marked_p (const struct Lisp_Marker *m);
 ^~~~~~
alloc.c:396:1: error: duplicate 'static'
 INLINE static void set_marker_marked (struct Lisp_Marker *m);
 ^~~~~~
alloc.c:398:1: error: duplicate 'static'
 INLINE static bool overlay_marked_p (const struct Lisp_Overlay *m);
 ^~~~~~
alloc.c:399:1: error: duplicate 'static'
 INLINE static void set_overlay_marked (struct Lisp_Overlay *m);
 ^~~~~~
alloc.c:401:1: error: duplicate 'static'
 INLINE static bool interval_marked_p (INTERVAL i);
 ^~~~~~
alloc.c:402:1: error: duplicate 'static'
 INLINE static void set_interval_marked (INTERVAL i);
 ^~~~~~
gmake[2]: *** [Makefile:386: alloc.o] Error 1

Regards,
-- 
Yoshiaki Kasahara
kasahara@nc.kyushu-u.ac.jp

Re: Preview: portable dumper

[Daniel Colascione]

On 02/14/2018 11:44 PM, Yoshiaki Kasahara wrote:
> Hello,
> 
> I cannot build pdumper branch on FreeBSD 11.1-STABLE with gcc 6.4.0.
> I guess INLINE macro already contains 'static' on FreeBSD.

Should be fixed.

Re: Preview: portable dumper

[Yoshiaki Kasahara]

On Thu, 15 Feb 2018 14:17:55 -0800,
	Daniel Colascione <dancol@dancol.org> said:

> On 02/14/2018 11:44 PM, Yoshiaki Kasahara wrote:
>> Hello,
>> I cannot build pdumper branch on FreeBSD 11.1-STABLE with gcc 6.4.0.
>> I guess INLINE macro already contains 'static' on FreeBSD.
> 
> Should be fixed.

Thank you.  I could build it and it seems working as expected.  I'm
writing this message using pdumper emacs.

Regards,
-- 
Yoshiaki Kasahara
kasahara@nc.kyushu-u.ac.jp

Re: Preview: portable dumper

[Andreas Schwab]

Doesn't work on powerpc:

../../emacs/src/pdumper.c:2479: Emacs fatal error: assertion failed: vector_nbytes ((struct Lisp_Vector *) in) == out_size

(gdb) p out_size
$1 = 536
(gdb) p vector_nbytes ((struct Lisp_Vector *) in)
$2 = 544

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Preview: portable dumper

[Daniel Colascione]

On 02/17/2018 02:31 AM, Andreas Schwab wrote:
> Doesn't work on powerpc:
> 
> ../../emacs/src/pdumper.c:2479: Emacs fatal error: assertion failed: vector_nbytes ((struct Lisp_Vector *) in) == out_size
> 
> (gdb) p out_size
> $1 = 536
> (gdb) p vector_nbytes ((struct Lisp_Vector *) in)
> $2 = 544
> 
> Andreas.
> 

Thanks for testing. I don't have a PowerPC handy. I guess I can go find 
an emulator, but in the meantime: can you think of anything specific to 
PowerPC that might break the code? Word size? pdumper works for me in 
both 32- and 64-bit forms on x86. Endian? AIUI, PowerPC has selectable 
endianness. Are you big- or little-endian? Besides, pdumper *should* be 
endian-neutral. And if we were looking at an endianness mismatch, I'd 
expect a much bigger mismatch than 544 vs. 536. What stack do you have 
there? Does your system use natural alignment?

Re: Preview: portable dumper

[Andreas Schwab]

On Feb 19 2018, Daniel Colascione <dancol@dancol.org> wrote:

> pdumper works for me in both 32- and 64-bit forms on x86.

That's pure luck.  There is nothing that forces the size of struct
buffer to be EMACS_INT aligned.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Preview: portable dumper

[Daniel Colascione]

[-- Attachment #1: Type: text/html, Size: 708 bytes --]

Re: Preview: portable dumper

[Andreas Schwab]

On Feb 19 2018, Daniel Colascione <dancol@dancol.org> wrote:

> <div dir='auto'><div><div class="gmail_extra"><div class="gmail_quote">On Feb 19, 2018 12:39 PM, Andreas Schwab &lt;schwab@linux-m68k.org&gt; wrote:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">On Feb 19 2018, Daniel Colascione &lt;dancol@dancol.org&gt; wrote:
> <br>
>
> <br>
> &gt; pdumper works for me in both 32- and 64-bit forms on x86.
> <br>
>
> <br>
> That's pure luck.&nbsp; There is nothing that forces the size of struct
> <br>
> buffer to be EMACS_INT aligned.
> <br></p></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">No, there isn't. That's why the code generally doesn't require that it be.</div></div>

Then you need to fix that.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Preview: portable dumper

[Daniel Colascione]

On 02/19/2018 01:41 PM, Andreas Schwab wrote:
> On Feb 19 2018, Daniel Colascione <dancol@dancol.org> wrote:
> 
>> <div dir='auto'><div><div class="gmail_extra"><div class="gmail_quote">On Feb 19, 2018 12:39 PM, Andreas Schwab &lt;schwab@linux-m68k.org&gt; wrote:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">On Feb 19 2018, Daniel Colascione &lt;dancol@dancol.org&gt; wrote:
>> <br>
>>
>> <br>
>> &gt; pdumper works for me in both 32- and 64-bit forms on x86.
>> <br>
>>
>> <br>
>> That's pure luck.&nbsp; There is nothing that forces the size of struct
>> <br>
>> buffer to be EMACS_INT aligned.
>> <br></p></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">No, there isn't. That's why the code generally doesn't require that it be.</div></div>
> 
> Then you need to fix that.

I will, but I don't understand why you've been so unhelpful.