bug#39962: 27.0.90; Crash in Emacs 27.0.90

[Pip Cet <pipcet@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3261 bytes --]

On Thu, Mar 12, 2020 at 6:13 PM Pieter van Oostrum
<pieter-l@vanoostrum.org> wrote:
> > My guess is 0x7ffeef270000 is your stack's guard page... Can you print
> > $rsp to confirm?
>
> Sorry, because of the erratic behaviour of GDB I killed that one. I have a new segfault in the GC. It is a long stack trace, so it could be a stack overflow. And, by the way, I had the two brakpoints set for the assignments to marker->charpos that Eli suggested, but they were not triggered. I have dumped a part of the stack trace below.

Thanks! I believe that solves it.

That indeed looks like a stack overflow.

Here's some speculation about what I think is happening:

We're seeing deep recursion in the garbage collector. If you look at
the tag bits of the objects marked by mark_object, you'll notice the
sequence is

symbol - cons - vectorlike - vectorlike - symbol - cons - vectorlike -
vectorlike - ...

That means there are thousands of symbols referring to values which
again contain symbols, and so on.

I suspect this code in vm-summary.el, or similar code, at least:

(defun vm-make-message ()
  "Create a new blank message struct."
  (let ((mvec (make-vector 5 nil))
    sym)
    (vm-set-softdata-of mvec (make-vector vm-softdata-vector-length nil))
    (vm-set-location-data-of
     mvec (make-vector vm-location-data-vector-length nil))
    (vm-set-mirror-data-of
     mvec (make-vector vm-mirror-data-vector-length nil))
    (vm-set-message-id-number-of mvec (int-to-string vm-message-id-number))
    (vm-increment vm-message-id-number)
    (vm-set-buffer-of mvec (current-buffer))
    ;; We use an uninterned symbol here as a level of indirection
    ;; from a purely self-referential structure.  This is
    ;; necessary so that Emacs debugger can be used on this
    ;; program.
    (setq sym (make-symbol "<<>>"))
    (set sym mvec)
    (vm-set-real-message-sym-of mvec sym)
    (vm-set-mirrored-message-sym-of mvec sym)
    ;; Another uninterned symbol for the virtual messages list.
    (setq sym (make-symbol "<v>"))
    (set sym nil)
    (vm-set-virtual-messages-sym-of mvec sym)
    ;; Another uninterned symbol for the reverse link
    ;; into the message list.
    (setq sym (make-symbol "<--"))
    (vm-set-reverse-link-sym-of mvec sym)
    mvec ))

Essentially, that code is building a singly-linked list of message
vectors, but the links go via symbols rather than directly to the next
message. The garbage collector isn't written for that case, and
recurses rather than iterating, causing the stack overflow.

The first attachment to this message is an Elisp file which does the
same thing, by creating thousands of symbols. On GNU/Linux, with
fairly default standard stack size settings, I get a segfault after
some 85,000 symbols have been created.

The second attachment is a patch which is
1. untested
2. a dirty workaround
3. not intended for inclusion in the master branch
4. not intended for inclusion in the emacs-27 branch.

It's possible this patch will work around the problem and result in a
different bug, or, less optimistically, fix this bug. With the patch,
I'm able to make it through the first 2^20 iterations of
symbol-crash.el without a segfault.

[-- Attachment #2: symbol-crash.el --]
[-- Type: text/x-emacs-lisp, Size: 222 bytes --]

(let* ((sym (make-symbol ""))
       (osym sym))
  (dotimes (i 1000000000)
    (set sym (make-symbol ""))
    (setq sym (symbol-value sym))
    (message "%d" i)
    (when (= 0 (logand i (1+ i)))
      (garbage-collect))))

[-- Attachment #3: 0001-recurse-into-symbol-values-rather-than-along-the-sym.patch --]
[-- Type: text/x-patch, Size: 1513 bytes --]

From d562402198a0341e5a1bbfff4f0e8ea0df63d484 Mon Sep 17 00:00:00 2001
From: Pip Cet <pipcet@gmail.com>
Date: Thu, 12 Mar 2020 19:47:10 +0000
Subject: [PATCH] recurse into symbol values rather than along the symbol chain
 when GCing

---
 src/alloc.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/src/alloc.c b/src/alloc.c
index 1c6b664b22..0db9620f3e 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -6581,9 +6581,19 @@ #define CHECK_ALLOCATED_AND_LIVE_SYMBOL()	((void) 0)
 	eassert (valid_lisp_object_p (ptr->u.s.function));
 	mark_object (ptr->u.s.function);
 	mark_object (ptr->u.s.plist);
+	if (!PURE_P (XSTRING (ptr->u.s.name)))
+          set_string_marked (XSTRING (ptr->u.s.name));
+        mark_interval_tree (string_intervals (ptr->u.s.name));
 	switch (ptr->u.s.redirect)
 	  {
-	  case SYMBOL_PLAINVAL: mark_object (SYMBOL_VAL (ptr)); break;
+	  case SYMBOL_PLAINVAL:
+	    if (!ptr->u.s.next)
+	      {
+		obj = SYMBOL_VAL (ptr);
+		goto loop;
+	      }
+	    mark_object (SYMBOL_VAL (ptr));
+	    break;
 	  case SYMBOL_VARALIAS:
 	    {
 	      Lisp_Object tem;
@@ -6602,9 +6612,6 @@ #define CHECK_ALLOCATED_AND_LIVE_SYMBOL()	((void) 0)
 	    break;
 	  default: emacs_abort ();
 	  }
-	if (!PURE_P (XSTRING (ptr->u.s.name)))
-          set_string_marked (XSTRING (ptr->u.s.name));
-        mark_interval_tree (string_intervals (ptr->u.s.name));
 	/* Inner loop to mark next symbol in this bucket, if any.  */
 	po = ptr = ptr->u.s.next;
 	if (ptr)
-- 
2.25.1