Tags: patch

Picking up on a side issue from Bug#30243:

>>> emacs: malloc.c:2427: sysmalloc: Assertion `(old_top == initial_top
>>> (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE &&
>>> prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1))
>>> == 0)' failed.
>>> Fatal error 6: Aborted

> The problem appears to be that we hit the limit in grow_specpdl(),
> and then call signal_error which does Ffuncall and then
> record_in_backtrace writes to specdl, this latter write is invalid
> since we failed to grow specdl before.  Thus memory corruption,
> undefined behaviour, etc.
>
> #0  0x000000000063999d in record_in_backtrace (function=XIL(0xd9ea380), args=0xffef5b188, nargs=2)
>     at ../../src/eval.c:2096
> #1  0x000000000063b8c9 in Ffuncall (nargs=3, args=0xffef5b180) at ../../src/eval.c:2746
> #2  0x000000000063b320 in call2 (fn=XIL(0xd9ea380), arg1=XIL(0x5250), arg2=XIL(0x1161fc03))
>     at ../../src/eval.c:2625
> #3  0x00000000006381db in signal_or_quit (error_symbol=XIL(0x5250), data=XIL(0x1161fc03), 
>     keyboard_quit=false) at ../../src/eval.c:1565
> #4  0x000000000063806d in Fsignal (error_symbol=XIL(0x5250), data=XIL(0x1161fc03))
>     at ../../src/eval.c:1514
> #5  0x000000000057939a in xsignal (error_symbol=XIL(0x5250), data=XIL(0x1161fc03))
>     at ../../src/lisp.h:3861
> #6  0x0000000000638704 in signal_error (s=0x75e388 "Variable binding depth exceeds max-specpdl-size", 
>     arg=XIL(0)) at ../../src/eval.c:1688
> #7  0x00000000006398cd in grow_specpdl () at ../../src/eval.c:2080
> (More stack frames follow...)

A simple reproducer from emacs -Q, C-u C-M-x on the following:

    (defun foo ()
      (let ((x 1))
        (foo)))

then evaluate (foo) and git 'g' to continue until the "Variable binding
depth exceeds max-specpdl-size" error.  At that point the memory
corruption has happened (verified with valgrind), although I found I had
to split window to actually trigger the malloc assertion.

The following patch solves the problem by not calling
signal-hook-function when the specpdl array is exhausted.  I think it
could be safe for emacs-26.