bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

[路客]

Hi all,

I found a short data clip can always crash Emacs 26.3 or 27.1.50 with
a terminal backtrace (core) dump in the "read()" function. It took me
a few hours to narrow it down and finally reach this minimal crashing
data set:

----- code begin -----
(#1=(#("000008964 .gnus.el" 0 18 (r #1#))
(def #2=#("000008964 .gnus.el" 0 18
(r
(#2#
(def #3=#("000006393 .gnus.el" 0 18
(r #4=(#3#
(def
#("000006393 .gnus.el" 0 18 (r #4#)) "x"))))"x"))))"x")))
----- code end -----

Try to `read' or `eval' this block of code (C-x C-e) will immediately
crash Emacs 26.3 or 27.1.50; however, older Emacs 26.0.50 works well
by entering the debugger with an error like:

----- elisp debugger message begin -----
Debugger entered--Lisp error: (invalid-function (#("000008964
.gnus.el" 0 18 (r #1)) (def #("000008964 .gnus.el" 0 18 (r (#3 (def
#("000006393 .gnus.el" 0 18 (r (#7 (def #("000006393 .gnus.el" 0 18 (r
#9)) "x")))) "x")))) "x")))
  ((#("000008964 .gnus.el" 0 18 (r #1)) (def #("000008964 .gnus.el" 0
18 (r (#3 (def #("000006393 .gnus.el" 0 18 ...) "x")))) "x")))
  eval(((#("000008964 .gnus.el" 0 18 (r #2)) (def #("000008964
.gnus.el" 0 18 (r (#4 (def #("000006393 .gnus.el" 0 18 ...) "x"))))
"x"))) nil)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)
----- elisp debugger message end -----

Funny thing is that if I change any of the above ".gnus.el" to another
file name, Emacs won't crash and is able to enter the debugger like
26.0.50. Is there anything special about the ".gnus.el" ?

This short crashing code block was actually a lot longer as a part of
a big bookmark file, and has long ago started to crash newer Emacs
than 26.0.50. I didn't know the root cause till these days I decided
to figure that out and finally extracted that block out and simplified
it to this simple form.

Could anyone help fix this long standing issue? Thanks.

-- 
Best regards,
Luke Lee

bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

[Eli Zaretskii]

> From: 路客 <luke.yx.lee@gmail.com>
> Date: Wed, 17 Feb 2021 18:13:56 +0800
> 
> ----- code begin -----
> (#1=(#("000008964 .gnus.el" 0 18 (r #1#))
> (def #2=#("000008964 .gnus.el" 0 18
> (r
> (#2#
> (def #3=#("000006393 .gnus.el" 0 18
> (r #4=(#3#
> (def
> #("000006393 .gnus.el" 0 18 (r #4#)) "x"))))"x"))))"x")))
> ----- code end -----
> 
> Try to `read' or `eval' this block of code (C-x C-e) will immediately
> crash Emacs 26.3 or 27.1.50; however, older Emacs 26.0.50 works well
> by entering the debugger with an error like:

It's an infinite recursion in substitute_object_recurse, called by
lread--substitute-object-in-subtree.

bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

[路客]

> It's an infinite recursion in substitute_object_recurse, called by
> lread--substitute-object-in-subtree.

I see, but why is Emacs 26.0.50 or earlier able to catch this issue?
Shouldn't the read() function try to prevent itself from crashing?

bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

[Eli Zaretskii]

> From: 路客 <luke.yx.lee@gmail.com>
> Date: Thu, 18 Feb 2021 09:56:06 +0800
> Cc: 46586@debbugs.gnu.org
> 
> > It's an infinite recursion in substitute_object_recurse, called by
> > lread--substitute-object-in-subtree.
> 
> I see, but why is Emacs 26.0.50 or earlier able to catch this issue?

The related code was refactored since then.  (And I'm not sure Emacs
26.0.50 indeed identified the problem correctly, see below.  So it
could be just sheer luck that it didn't crash back then.)

> Shouldn't the read() function try to prevent itself from crashing?

It should, so this is a bug.

But how did such a form get originated?  It looks like it's indeed
self-referential, and thus is got to trigger infinite recursion:

> (#1=(#("000008964 .gnus.el" 0 18 (r #1#))
> (def #2=#("000008964 .gnus.el" 0 18
> (r
> (#2#
> (def #3=#("000006393 .gnus.el" 0 18
> (r #4=(#3#
> (def
> #("000006393 .gnus.el" 0 18 (r #4#)) "x"))))"x"))))"x")))

The last part references itself: it seems to define a string with a
text property that is the same string.

Stepping through the code in substitute_object_recurse, I see that we
end up recursively expanding this string:

  #("000006393 .gnus.el" 0 18 (r (#("000006393 .gnus.el" 0 18 (r #2)) (def #0 "x"))))

Which then yields this:

  #("000006393 .gnus.el" 0 18 (r (#0 (def #("000006393 .gnus.el" 0 18 (r #2)) "x"))))

And that again yields

  #("000006393 .gnus.el" 0 18 (r (#("000006393 .gnus.el" 0 18 (r #2)) (def #0 "x"))))

Etc. etc., ad nauseam (or, rather, until we exhaust the C run-time
stack and segfault).

Does anyone see how to stop this infinite recursion, except by
counting recursive invocation levels and bailing out at some arbitrary
depth?

bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

[Andreas Schwab]

On Feb 18 2021, Eli Zaretskii wrote:

> Does anyone see how to stop this infinite recursion, except by
> counting recursive invocation levels and bailing out at some arbitrary
> depth?

Shouldn't the subst->seen list prevent recursion?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: 路客 <luke.yx.lee@gmail.com>,  46586@debbugs.gnu.org,
>   Stefan Monnier
>  <monnier@iro.umontreal.ca>
> Date: Thu, 18 Feb 2021 15:45:01 +0100
> 
> On Feb 18 2021, Eli Zaretskii wrote:
> 
> > Does anyone see how to stop this infinite recursion, except by
> > counting recursive invocation levels and bailing out at some arbitrary
> > depth?
> 
> Shouldn't the subst->seen list prevent recursion?

Maybe, but the insertion into the 'seen' list is conditioned on #n=
being in 'completed'.  Or maybe I don't understand the logic of the
code well enough.

bug#46586: 26.3, 27.1.50; Emacs crash in a backtrace (core) dump (a long standing issue)

[Lars Ingebrigtsen]

路客 <luke.yx.lee@gmail.com> writes:

> I found a short data clip can always crash Emacs 26.3 or 27.1.50 with
> a terminal backtrace (core) dump in the "read()" function. It took me
> a few hours to narrow it down and finally reach this minimal crashing
> data set:
>
> ----- code begin -----
> (#1=(#("000008964 .gnus.el" 0 18 (r #1#))
> (def #2=#("000008964 .gnus.el" 0 18
> (r
> (#2#
> (def #3=#("000006393 .gnus.el" 0 18
> (r #4=(#3#
> (def
> #("000006393 .gnus.el" 0 18 (r #4#)) "x"))))"x"))))"x")))
> ----- code end -----
>
> Try to `read' or `eval' this block of code (C-x C-e) will immediately
> crash Emacs 26.3 or 27.1.50; however, older Emacs 26.0.50 works well
> by entering the debugger with an error like:

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

I can reproduce this problem in Emacs 28.1, but not in Emacs 29 -- the
reader/printer/gc was recently rewritten to handle recursive structures
better, so I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no