bug#31919: 26.1.50; Lisp Debugger doesn't work when at stack limit

bug#31919: 26.1.50; Lisp Debugger doesn't work when at stack limit

[Gemini Lasswell]

[-- Attachment #1: Type: text/plain, Size: 1804 bytes --]

When max-lisp-eval-depth is exceeded and that error invokes the Lisp
Debugger, it prints an error message, doesn't produce a backtrace, and
leaves Emacs in a borked state.  In Emacs 25.3.1, the Debugger works
in this situation although the rest of Emacs doesn't work well due to
being near the stack limit, but you can exit the Debugger and things
will go back to normal.

The reason for the regression is that backtrace printing is now done
in Lisp, in cl-print, instead of in C.  If cl-print isn't yet
loaded, being at the stack limit can cause it to fail to load.

To reproduce, from emacs -Q, enter this code into a buffer and
evaluate it:

(toggle-debug-on-error)
(defun my-func (arg)
  (+ (length arg) (my-func arg)))
(my-func "hello")

Result:

Instead of the *Backtrace* buffer, Emacs instead shows *Compile-Log*,
which in my case contains:

/nix/store/s6ji5vn6jbsjpp6wwbix2daigkcsvj7h-emacs-26.1.50/share/emacs/26.1.50/lisp/emacs-lisp/cl-print.elc:Error: Lisp nesting exceeds ‘max-lisp-eval-depth’

In the echo area, this message appears:

cl--generic-make-next-function: Symbol’s function definition is void: t

Emacs is at this point barely usable because it is inside the
debugger's recursive edit and at its stack limit, making
max-lisp-eval-depth errors easy to encounter.  The *Backtrace* buffer
exists but is empty and in Fundamental mode, so you can't use it to
quit the debugger.  You can recover from the situation with
M-x top-level RET.

If cl-print is already loaded the above example will work, but if you
evaluate the following, the Debugger buffer will appear with a
truncated backtrace:

(my-func '(1 (2 (3 (4 (5 (6 (7 (8)))))))))

Here's a patch to give the Debugger and cl-print more stack space
during the recursive edit:


[-- Attachment #2: 0001-Increase-max-lisp-eval-depth-adjustment-while-in-deb.patch --]
[-- Type: text/plain, Size: 1066 bytes --]

From d044dc12a2b5794bd1155fd5b7ff7adb3bc8841d Mon Sep 17 00:00:00 2001
From: Gemini Lasswell <gazally@runbox.com>
Date: Wed, 20 Jun 2018 13:58:33 -0700
Subject: [PATCH] Increase max-lisp-eval-depth adjustment while in debugger

* src/eval.c (call_debugger): Increase the amount of extra stack
depth given to the debugger to allow it to call cl-print.
---
 src/eval.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/eval.c b/src/eval.c
index ca1eb84ff3..f9bc13ade7 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -282,8 +282,8 @@ call_debugger (Lisp_Object arg)
   /* Do not allow max_specpdl_size less than actual depth (Bug#16603).  */
   EMACS_INT old_max = max (max_specpdl_size, count);
 
-  if (lisp_eval_depth + 40 > max_lisp_eval_depth)
-    max_lisp_eval_depth = lisp_eval_depth + 40;
+  if (lisp_eval_depth + 80 > max_lisp_eval_depth)
+    max_lisp_eval_depth = lisp_eval_depth + 80;
 
   /* While debugging Bug#16603, previous value of 100 was found
      too small to avoid specpdl overflow in the debugger itself.  */
-- 
2.16.4

bug#31919: 26.1.50; Lisp Debugger doesn't work when at stack limit

[Gemini Lasswell]

Is this patch OK for emacs-26?

> Here's a patch to give the Debugger and cl-print more stack space
> during the recursive edit:
>
>>From d044dc12a2b5794bd1155fd5b7ff7adb3bc8841d Mon Sep 17 00:00:00 2001
> From: Gemini Lasswell <gazally@runbox.com>
> Date: Wed, 20 Jun 2018 13:58:33 -0700
> Subject: [PATCH] Increase max-lisp-eval-depth adjustment while in debugger
>
> * src/eval.c (call_debugger): Increase the amount of extra stack
> depth given to the debugger to allow it to call cl-print.
> ---
>  src/eval.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/src/eval.c b/src/eval.c
> index ca1eb84ff3..f9bc13ade7 100644
> --- a/src/eval.c
> +++ b/src/eval.c
> @@ -282,8 +282,8 @@ call_debugger (Lisp_Object arg)
>    /* Do not allow max_specpdl_size less than actual depth (Bug#16603).  */
>    EMACS_INT old_max = max (max_specpdl_size, count);
>  
> -  if (lisp_eval_depth + 40 > max_lisp_eval_depth)
> -    max_lisp_eval_depth = lisp_eval_depth + 40;
> +  if (lisp_eval_depth + 80 > max_lisp_eval_depth)
> +    max_lisp_eval_depth = lisp_eval_depth + 80;
>  
>    /* While debugging Bug#16603, previous value of 100 was found
>       too small to avoid specpdl overflow in the debugger itself.  */

bug#31919: 26.1.50; Lisp Debugger doesn't work when at stack limit

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Date: Wed, 27 Jun 2018 10:16:57 -0700
> 
> Is this patch OK for emacs-26?

Yes, but please add a comment there describing the reason.  How much
more depth out of 40 is actually needed to allow cl-print calls, and
how much is safety margin?

bug#31919: 26.1.50; Lisp Debugger doesn't work when at stack limit

[Gemini Lasswell]

[-- Attachment #1: Type: text/plain, Size: 490 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, but please add a comment there describing the reason.  How much
> more depth out of 40 is actually needed to allow cl-print calls, and
> how much is safety margin?

I determined by experiment that 77 needs to be added to
max-lisp-eval-depth to permit the debugger to print
((1 (2 (3 (4 (5 (6 (7 (8))))))))).  So I changed the increment to 100.
But I really have no idea what is reasonable for a safety margin.
Here's a new patch with comments.


[-- Attachment #2: 0001-Increase-max-lisp-eval-depth-adjustment-while-in-deb.patch --]
[-- Type: text/plain, Size: 2010 bytes --]

From ab293d12ef045042b62df7670cf9fe05f175ce19 Mon Sep 17 00:00:00 2001
From: Gemini Lasswell <gazally@runbox.com>
Date: Wed, 20 Jun 2018 13:58:33 -0700
Subject: [PATCH] Increase max-lisp-eval-depth adjustment while in debugger

* src/eval.c (call_debugger): Increase the amount of extra Lisp
evaluation depth given to the debugger to allow it to call cl-print.
* lisp/emacs-lisp/debug.el (debugger-setup-buffer): Add a comment
to suggest updating call_debugger when changing print-level.
---
 lisp/emacs-lisp/debug.el | 1 +
 src/eval.c               | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 593fab9727..821d674882 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -322,6 +322,7 @@ debugger-setup-buffer
                  (backtrace-frames 'debug)))
         (print-escape-newlines t)
         (print-escape-control-characters t)
+        ;; If you increase print-level, add more depth in call_debugger.
         (print-level 8)
         (print-length 50)
         (pos (point)))
diff --git a/src/eval.c b/src/eval.c
index ca1eb84ff3..40cba3bb1c 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -282,8 +282,12 @@ call_debugger (Lisp_Object arg)
   /* Do not allow max_specpdl_size less than actual depth (Bug#16603).  */
   EMACS_INT old_max = max (max_specpdl_size, count);
 
-  if (lisp_eval_depth + 40 > max_lisp_eval_depth)
-    max_lisp_eval_depth = lisp_eval_depth + 40;
+  /* The previous value of 40 is too small now that the debugger
+     prints using cl-prin1 instead of prin1.  Printing lists nested 8
+     deep (which is the value of print-level used in the debugger)
+     currently requires 77 additional frames.  See bug#31919.  */
+  if (lisp_eval_depth + 100 > max_lisp_eval_depth)
+    max_lisp_eval_depth = lisp_eval_depth + 100;
 
   /* While debugging Bug#16603, previous value of 100 was found
      too small to avoid specpdl overflow in the debugger itself.  */
-- 
2.16.4

bug#31919: 26.1.50; Lisp Debugger doesn't work when at stack limit

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Cc: 31919@debbugs.gnu.org
> Date: Wed, 27 Jun 2018 21:26:12 -0700
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Yes, but please add a comment there describing the reason.  How much
> > more depth out of 40 is actually needed to allow cl-print calls, and
> > how much is safety margin?
> 
> I determined by experiment that 77 needs to be added to
> max-lisp-eval-depth to permit the debugger to print
> ((1 (2 (3 (4 (5 (6 (7 (8))))))))).  So I changed the increment to 100.
> But I really have no idea what is reasonable for a safety margin.
> Here's a new patch with comments.

Thanks, this LGTM for emacs-26.