gdb doesn't print Lisp backtrace in some circumstances.

gdb doesn't print Lisp backtrace in some circumstances.

[Alan Mackenzie]

Hello, Emacs.

Yesterday I got a core dump from a segmentation fault in Emacs (my
development version, not master).  I loaded this into gdb inside Emacs
to have a look at it.

The backtrace command output the C data as it should, but on coming to
the Lisp backtrace gave an error message about there needing to be a
process running to "do this".  (I don't have the exact message any
more.)  It would seem these fancy Lisp facilities only work when the
process that produced them is still running.

All the information required to produce this Lisp backtrace is present
in the core dump.  Would it be possible, perhaps, to modify our .gdbinit
to use the running Emacs to process the core dump, or something like
that?  Extracting useful information from the C backtrace is slow and
tedious in the extreme.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Eli Zaretskii]

> Date: Fri, 12 Apr 2024 10:31:53 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> Yesterday I got a core dump from a segmentation fault in Emacs (my
> development version, not master).  I loaded this into gdb inside Emacs
> to have a look at it.
> 
> The backtrace command output the C data as it should, but on coming to
> the Lisp backtrace gave an error message about there needing to be a
> process running to "do this".  (I don't have the exact message any
> more.)  It would seem these fancy Lisp facilities only work when the
> process that produced them is still running.

Yes, because they call functions inside Emacs to format Lisp objects.

> All the information required to produce this Lisp backtrace is present
> in the core dump.  Would it be possible, perhaps, to modify our .gdbinit
> to use the running Emacs to process the core dump, or something like
> that?

I don't think so, no.  But you can reproduce the Lisp backtrace
manually (albeit tediously, by going over all the calls to Ffuncall,
eval_sub and suchlikes, and displaying their arg[0].  If it's a
symbol, typing xsymbol should show you the Lisp function being called.
If it is not a symbol, you could use xcar/xcdr etc., but that is much
more tedious, and I usually give up on those frames in the stack.

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Alan Mackenzie]

Hello, Eli.

On Fri, Apr 12, 2024 at 14:11:04 +0300, Eli Zaretskii wrote:
> > Date: Fri, 12 Apr 2024 10:31:53 +0000
> > From: Alan Mackenzie <acm@muc.de>

> > Yesterday I got a core dump from a segmentation fault in Emacs (my
> > development version, not master).  I loaded this into gdb inside Emacs
> > to have a look at it.

> > The backtrace command output the C data as it should, but on coming to
> > the Lisp backtrace gave an error message about there needing to be a
> > process running to "do this".  (I don't have the exact message any
> > more.)  It would seem these fancy Lisp facilities only work when the
> > process that produced them is still running.

> Yes, because they call functions inside Emacs to format Lisp objects.

> > All the information required to produce this Lisp backtrace is present
> > in the core dump.  Would it be possible, perhaps, to modify our .gdbinit
> > to use the running Emacs to process the core dump, or something like
> > that?

> I don't think so, no.

That's what I feared.

> But you can reproduce the Lisp backtrace manually (albeit tediously,
> by going over all the calls to Ffuncall, eval_sub and suchlikes, and
> displaying their arg[0].  If it's a symbol, typing xsymbol should show
> you the Lisp function being called.  If it is not a symbol, you could
> use xcar/xcdr etc., but that is much more tedious, and I usually give
> up on those frames in the stack.

I spent a large part of yesterday on this dump.  It ends up with a call
(apply apply ...) which goes into an infinite recursion with the number
of arguments increasing by 1 at each stage.  I've found the point in the
backtrace where the recursion starts, but it's tedious indeed getting
information out of it.  I think I'll have to, though, since I can't see
any other way of debugging this at the moment.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Eli Zaretskii]

> Date: Fri, 12 Apr 2024 11:29:29 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > But you can reproduce the Lisp backtrace manually (albeit tediously,
> > by going over all the calls to Ffuncall, eval_sub and suchlikes, and
> > displaying their arg[0].  If it's a symbol, typing xsymbol should show
> > you the Lisp function being called.  If it is not a symbol, you could
> > use xcar/xcdr etc., but that is much more tedious, and I usually give
> > up on those frames in the stack.
> 
> I spent a large part of yesterday on this dump.  It ends up with a call
> (apply apply ...) which goes into an infinite recursion with the number
> of arguments increasing by 1 at each stage.  I've found the point in the
> backtrace where the recursion starts, but it's tedious indeed getting
> information out of it.  I think I'll have to, though, since I can't see
> any other way of debugging this at the moment.

One way of avoiding all this tedious manual work is to try to catch
the problem with a breakpoint in a running Emacs, not in a core dump.
You need to be creative and most probably invest a lot of efforts in
trial-and-error to come up with a breakpoint place and condition that
on one hand doesn't cause it to trigger too often for false positives,
and OTOH doesn't miss the real problem.

An easier way is to run Emacs under GDB to begin with, and do what you
need to do when it catches the segfault, before dumping core.  At that
point, you still have a running Emacs, and can call functions inside
it, albeit less safely (because when Emacs is crashing, its Lisp
machine could be utterly confused or messed up).

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Alan Mackenzie]

Hello, Eli.

On Fri, Apr 12, 2024 at 15:03:03 +0300, Eli Zaretskii wrote:
> > Date: Fri, 12 Apr 2024 11:29:29 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>
> > 
> > > But you can reproduce the Lisp backtrace manually (albeit tediously,
> > > by going over all the calls to Ffuncall, eval_sub and suchlikes, and
> > > displaying their arg[0].  If it's a symbol, typing xsymbol should show
> > > you the Lisp function being called.  If it is not a symbol, you could
> > > use xcar/xcdr etc., but that is much more tedious, and I usually give
> > > up on those frames in the stack.
> > 
> > I spent a large part of yesterday on this dump.  It ends up with a call
> > (apply apply ...) which goes into an infinite recursion with the number
> > of arguments increasing by 1 at each stage.  I've found the point in the
> > backtrace where the recursion starts, but it's tedious indeed getting
> > information out of it.  I think I'll have to, though, since I can't see
> > any other way of debugging this at the moment.

> One way of avoiding all this tedious manual work is to try to catch
> the problem with a breakpoint in a running Emacs, not in a core dump.

That won't work for this situation.  The seg fault is happening whilst
loading isearch.el from loadup.el during early building.  I don't have a
running Emacs to probe it with.

> You need to be creative and most probably invest a lot of efforts in
> trial-and-error to come up with a breakpoint place and condition that
> on one hand doesn't cause it to trigger too often for false positives,
> and OTOH doesn't miss the real problem.

Yes.  This is the curse of using debuggers, the difficulty of getting
breakpoints placed effectively.  GDB is quite good in this respect,
since you can put a condition on any breakpoint to reduce the false
positives.

> An easier way is to run Emacs under GDB to begin with, and do what you
> need to do when it catches the segfault, before dumping core.  At that
> point, you still have a running Emacs, and can call functions inside
> it, albeit less safely (because when Emacs is crashing, its Lisp
> machine could be utterly confused or messed up).

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Eli Zaretskii]

> Date: Fri, 12 Apr 2024 13:18:15 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > One way of avoiding all this tedious manual work is to try to catch
> > the problem with a breakpoint in a running Emacs, not in a core dump.
> 
> That won't work for this situation.  The seg fault is happening whilst
> loading isearch.el from loadup.el during early building.  I don't have a
> running Emacs to probe it with.

I don't see why can't you have a running Emacs.  You just need to run
temacs under GDB instead of emacs, and you need to run the exact
command that crashes under GDB.  To see what command crashes, run
"make V=1", to force make to display every command it runs in its
entirety.

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 12 Apr 2024 10:31:53 +0000
>> From: Alan Mackenzie <acm@muc.de>
>> 
>> Yesterday I got a core dump from a segmentation fault in Emacs (my
>> development version, not master).  I loaded this into gdb inside Emacs
>> to have a look at it.
>> 
>> The backtrace command output the C data as it should, but on coming to
>> the Lisp backtrace gave an error message about there needing to be a
>> process running to "do this".  (I don't have the exact message any
>> more.)  It would seem these fancy Lisp facilities only work when the
>> process that produced them is still running.
>
> Yes, because they call functions inside Emacs to format Lisp objects.
>
>> All the information required to produce this Lisp backtrace is present
>> in the core dump.  Would it be possible, perhaps, to modify our .gdbinit
>> to use the running Emacs to process the core dump, or something like
>> that?
>
> I don't think so, no.  But you can reproduce the Lisp backtrace
> manually (albeit tediously, by going over all the calls to Ffuncall,
> eval_sub and suchlikes, and displaying their arg[0].  If it's a
> symbol, typing xsymbol should show you the Lisp function being called.
> If it is not a symbol, you could use xcar/xcdr etc., but that is much
> more tedious, and I usually give up on those frames in the stack.

I think it's easier to go through the specbinding stack
(current_thread->m_specdl). For LLDB, I did that in the xbacktrace
function in etc/emacs_lldb.py.

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> I think it's easier to go through the specbinding stack
> (current_thread->m_specdl). For LLDB, I did that in the xbacktrace
> function in etc/emacs_lldb.py.

It just remembered that the very first xbacktrace I added did it in a
similar way, so a very old .gdbinit may contain something of a starting
point.

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Alan Mackenzie <acm@muc.de>,  emacs-devel@gnu.org
> Date: Fri, 12 Apr 2024 16:56:58 +0200
> 
> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> 
> > I think it's easier to go through the specbinding stack
> > (current_thread->m_specdl). For LLDB, I did that in the xbacktrace
> > function in etc/emacs_lldb.py.
> 
> It just remembered that the very first xbacktrace I added did it in a
> similar way, so a very old .gdbinit may contain something of a starting
> point.

Printing Lisp objects would still be a problem, though, even if the
backtrace is shown.

Re: gdb doesn't print Lisp backtrace in some circumstances.

[Alan Mackenzie]

Hello, Eli.

On Fri, Apr 12, 2024 at 16:24:53 +0300, Eli Zaretskii wrote:
> > Date: Fri, 12 Apr 2024 13:18:15 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > One way of avoiding all this tedious manual work is to try to catch
> > > the problem with a breakpoint in a running Emacs, not in a core dump.

> > That won't work for this situation.  The seg fault is happening whilst
> > loading isearch.el from loadup.el during early building.  I don't have a
> > running Emacs to probe it with.

> I don't see why can't you have a running Emacs.  You just need to run
> temacs under GDB instead of emacs, and you need to run the exact
> command that crashes under GDB.  To see what command crashes, run
> "make V=1", to force make to display every command it runs in its
> entirety.

Maybe I'll try that.  What I eventually did yesterday was to code in a
limit of 150 to the length of the last argument in Fapply, signalling an
error when it was higher.  Even this (Lisp) backtrace is difficult to
decypher.

The backtrace I got is a mess.  The problem has a lot to do with generic
functions, and cl-generic.el is as good as entirely undocumented.
Didn't we have somebody volunteering to fix this some months ago?  Has
anything come of that, yet?

-- 
Alan Mackenzie (Nuremberg, Germany).