unexec problem

unexec problem

[Richard Stallman]

When I try to bootstrap, bootstrap-emacs crashes
in the first call to malloc.  Experience shows this is
usually due to a bug in the unexec mechanism.
However, unexelf.c is unchanged from Emacs 22.

Since this sort of thing is a big pain to debug,
does anyone have any clues?

Re: unexec problem

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

> When I try to bootstrap, bootstrap-emacs crashes
> in the first call to malloc.  Experience shows this is
> usually due to a bug in the unexec mechanism.
> However, unexelf.c is unchanged from Emacs 22.
>
> Since this sort of thing is a big pain to debug,
> does anyone have any clues?

Is this on the trunk?

Re: unexec problem

[Richard Stallman]

    > When I try to bootstrap, bootstrap-emacs crashes
    > in the first call to malloc.  Experience shows this is
    > usually due to a bug in the unexec mechanism.
    > However, unexelf.c is unchanged from Emacs 22.
    >
    > Since this sort of thing is a big pain to debug,
    > does anyone have any clues?

    Is this on the trunk?

Yes.  And it worked a week ago.
I think the unicode merge broke it.

Re: unexec problem

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Tue, 05 Feb 2008 03:04:25 -0500
> Cc: emacs-devel@gnu.org
> 
>     > When I try to bootstrap, bootstrap-emacs crashes
>     > in the first call to malloc.  Experience shows this is
>     > usually due to a bug in the unexec mechanism.
>     > However, unexelf.c is unchanged from Emacs 22.
>     >
>     > Since this sort of thing is a big pain to debug,
>     > does anyone have any clues?
> 
>     Is this on the trunk?
> 
> Yes.  And it worked a week ago.
> I think the unicode merge broke it.

Probably.

How about running the bootstrap temacs (the one that is dumped to
produce bootstrap-emacs) under GDB with a breakpoint in malloc, and
recording all calls to malloc with their callers?  Then you could
compare that with a similar run for a pre-merge CVS, and look for some
callers that could potentially lead to corruption of the malloc chain
the the dumped executable.

Re: unexec problem

[Richard Stallman]

    How about running the bootstrap temacs (the one that is dumped to
    produce bootstrap-emacs) under GDB with a breakpoint in malloc, and
    recording all calls to malloc with their callers?

I don't know how to do that.  Can you tell me how?

Re: unexec problem

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: cyd@stupidchicken.com, emacs-devel@gnu.org
> Date: Wed, 06 Feb 2008 11:12:02 -0500
> 
>     How about running the bootstrap temacs (the one that is dumped to
>     produce bootstrap-emacs) under GDB with a breakpoint in malloc, and
>     recording all calls to malloc with their callers?
> 
> I don't know how to do that.  Can you tell me how?

Something like this:

  cd src
  gdb ./temacs
  .....
  (gdb) set logging on
  (gdb) break malloc
  (gdb) commands
   > bt
   > continue
   > end
  (gdb) run --batch --load loadup bootstrap

Then, when temacs exits, you will have all the hits of this
breakpoint, together with the backtrace of each one of them, in the
file gdb.txt.  (You can change the name of the file with the command
"set logging file FOO".)

Re: unexec problem

[Richard Stallman]

      cd src
      gdb ./temacs
      .....
      (gdb) set logging on
      (gdb) break malloc
      (gdb) commands
       > bt
       > continue
       > end
      (gdb) run --batch --load loadup bootstrap

I tried that.  After it had run for several hours and produced 190 meg
of data, I killed it.  I deleted the file because I can't possibly
see any pattern in such a big collection of data except by some
automated method, and we don't know what it should look for.

If you have a concrete idea for how to analyze that data,
I could run it again.  Maybe overnight it would finish.

Re: unexec problem

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: emacs-devel@gnu.org
> Date: Thu, 07 Feb 2008 23:15:34 -0500
> 
>       cd src
>       gdb ./temacs
>       .....
>       (gdb) set logging on
>       (gdb) break malloc
>       (gdb) commands
>        > bt
>        > continue
>        > end
>       (gdb) run --batch --load loadup bootstrap
> 
> I tried that.  After it had run for several hours and produced 190 meg
> of data, I killed it.  I deleted the file because I can't possibly
> see any pattern in such a big collection of data except by some
> automated method, and we don't know what it should look for.

Maybe "bt N", where N is some small number, will produce a more
manageable file.

> If you have a concrete idea for how to analyze that data,
> I could run it again.

The idea was that maybe some library function we are now calling, and
that calls malloc internally, causes the corruption of the memory
allocation in the dumped Emacs.  I don't know how to spot such a
function except by the method I suggested.

Re: unexec problem

[Richard Stallman]

    The idea was that maybe some library function we are now calling, and
    that calls malloc internally, causes the corruption of the memory
    allocation in the dumped Emacs.  I don't know how to spot such a
    function except by the method I suggested.

The problem is how to find a candidate for the problem out of
thousands of backtraces of malloc calls.  I don't
see how I could even try.

Re: unexec problem

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: emacs-devel@gnu.org
> Date: Sun, 10 Feb 2008 13:42:09 -0500
> 
>     The idea was that maybe some library function we are now calling, and
>     that calls malloc internally, causes the corruption of the memory
>     allocation in the dumped Emacs.  I don't know how to spot such a
>     function except by the method I suggested.
> 
> The problem is how to find a candidate for the problem out of
> thousands of backtraces of malloc calls.

Does the dumped Emacs crash even if you dump it immediately after
temacs starts, without loading any of the Lisp files it loads during
loading loadup.el (by hacking it to comment out all the load
commands)?  If so, perhaps the amount of calls to malloc would be much
smaller, and you could look at it.

Re: unexec problem

[Richard Stallman]

    Does the dumped Emacs crash even if you dump it immediately after
    temacs starts, without loading any of the Lisp files it loads during
    loading loadup.el (by hacking it to comment out all the load
    commands)?  If so, perhaps the amount of calls to malloc would be much
    smaller, and you could look at it.

It crashes on the first call to malloc.  That means the data
structures as dumped must be corrupted.  But I don't know what the
data structures of malloc look like, not nowadays.

Re: unexec problem

[Dan Nicolaescu]

Richard Stallman <rms@gnu.org> writes:

  > When I try to bootstrap, bootstrap-emacs crashes
  > in the first call to malloc.  Experience shows this is
  > usually due to a bug in the unexec mechanism.
  > However, unexelf.c is unchanged from Emacs 22.
  > 
  > Since this sort of thing is a big pain to debug,
  > does anyone have any clues?

Have you tried building after doing a "make distclean"?
In the past I have seen weird unexec problems that went away after doing
a clean build...

Re: unexec problem

[Richard Stallman]

    Have you tried building after doing a "make distclean"?

I tried that just now, but it failed the same way.