bug#5662: flet not undone on lisp nesting error

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

> The cause of problem is that `unwind-forms’ will call the `eval_sub’
> function.  Of course, the `eval_sub’ function check for infinite
> recursion.  Consequently, the `unwind-protect’ wouldn't evaluate.

Thanks for digging into it.  Indeed, I think I see it now: in
`unbind_to' we run the unwind_forms without updating lisp_eval_depth, so
they're run with the lisp_eval_depth of the place where the signal
was thrown, so if the signal was thrown because lisp_eval_depth >
max_lisp_eval_depth, then the unwind forms will all fail by signaling

   error ("Lisp nesting exceeds `max-lisp-eval-depth'").

This said, at the time we run those unwind forms, the C stack is still
at the same "very deep" state as when the error was signaled, so the
lisp_eval_depth value is not completely wrong (more specifically, it
would be unwise in general to simply set it to the value corresponding
to the target of the longjmp, even though in your particular case you
could argue that it would be right).

So maybe the best solution would be to temporarily increase
max-lisp-eval-depth like we do for call_debugger.  Not sure how best to
do that, since the way it's done in call_debugger could have too high
a performance cost (it's not a problem for call_debugger since that's
an exceptional situation).


        Stefan


> You must be noted step 7~10.
> Problem's steps:
>   1. eval.c:1208:Funwind_protect:  record_unwind_protect (unwind_body, XCDR (args)); 
>   2. eval.c:1209:Funwind_protect:  val = eval_sub (XCAR (args)); 
>   3. eval.c:2090:eval_sub:  if (lisp_eval_depth > max_lisp_eval_depth) error ("Lisp nesting exceeds `max-lisp-eval-depth'”); 
>   4. eval.c:1582:xsignal:  Fsignal (error_symbol, data); 
>   5. eval.c:1558:Fsignal:  unwind_to_catch (h, unwind_data); 
>   6. eval.c:1161:unwind_to_catch:  unbind_to (handlerlist->pdlcount, Qnil); 
>   7. eval.c:3305:unbind_to:  unwind_body // specpdl_ptr->unwind_ptr.func (specpdl_ptr->unwind_ptr.arg);
>   8. eval.c:476:unwind_body:  Fprogn (body); 
>   9. eval.c:462:Fprogn:  val = eval_sub (XCAR (body)); 
>   10. eval.c:2090:eval_sub:  if (lisp_eval_depth > max_lisp_eval_depth) error ("Lisp nesting exceeds `max-lisp-eval-depth’”)


> You can avoid this problem using `ignore-errors’ macro.
> (eval.c:1175:unwind_to_catch:  lisp_eval_depth = catch->lisp_eval_depth;)

> ```elisp
> (defun endless-recursive ()
>   (endless-recursive))

> (unwind-protect
>     (ignore-errors (endless-recursive))
>   (message "finally"))
> ```

>> On Mar 6, 2010, at 5:30 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> 
>>> If I trigger a lisp nesting error with an infinite recursion inside a
>>> let and an flet binding, then the effects of the flet are not undone,
>>> resulting in a change of binding at the top-level.
>> 
>>> (defun g () 'g-orig)
>>> (setq a 'a-orig)
>> 
>>> (defun h ()
>>> (let ((a 'a-new))
>>> (flet ((g () 'g-new))
>>> (h))))
>>> (h)   ;; <-- Lisp nesting exceeds `max-lisp-eval-depth'
>>> (g)   ;; g-new ! 
>>> a     ;; a-orig 
>> 
>> I can indeed reproduce it.  I'm not sure yet what's going on, but it
>> might be due to the "Lisp nesting exceeds `max-lisp-eval-depth'" error
>> happening in the unwind-protect code (i.e. while undoing the `g'
>> binding).  The explanation can't be quite so simple, but my gut feeling
>> tells me it's got to do with it.
>> 
>> 
>> Stefan
>> 
>> 
>> 
>>