The poor quality of Emacs's backtraces

[Alan Mackenzie <acm@muc.de>]

Hello, Emacs.

On running the test suite (make check) on my development version, I
recently got lots of errors and backtraces in the files.log.  This is
fair enough.

What's not good is the message that precedes (or, in the test suite
follows) the backtrace.  In the current instance, it looks like:

   Test test-kill-buffer-auto-save-delete-no condition:
    (wrong-type-argument listp
                         #[257 "\300\211\2!\262\1\207" [yes-or-no-p] 4
                               "\n\n(fn ARG124 &optional)" t])
   FAILED  376/406  test-kill-buffer-auto-save-delete-no (0.012664 sec)

So, this says that _something_ wasn't a list, without telling me what the
something was.  It may have been the value of a variable, or the result
returned from evaluating a form, but the message gives no clue.

It says "wrong-type-argument", but doesn't say why it's wrong.  It
doesn't disclose which primitive detected the fault, though Emacs could
easily do this.

It doesn't make any effort to say _where_ the bug was detected.  If
you're lucky, it might be somewhere in the first function reported in the
backtrace, but only if there aren't any frivolous condition-case's in the
code which render any remaining backtrace useless.  But even if you know
what function it's in, unless that function is short, you don't know
_where_ in that function the error has occurred.  The Emacs backtrace
functions don't output any coordinates.  This could be done easily, for
example by printing the current position and the total length for a
compiled function (something like 56/120), or by outputting neighbouring
forms for an interpreted function.  This would be helpful information for
debugging.

As already said, the actual backtrace itself is often of little use, due
to the frivolous use of condition-case's.  That's even if you stop the
test suite truncating every line at ~70 characters.  (Why is this done?)
Currently the first few lines of my backtrace look like:

    Test test-kill-buffer-auto-save-delete-no backtrace:
  {comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>)((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
  apply({comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>) (lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil)
  comp-spill-lap-function((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
  comp-spill-lap((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
  comp--native-compile((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil "/tmp/test-nativecomp-cache-Dmx7GP/30.0.50-7a56150c...")
  comp-trampoline-compile(yes-or-no-p)
  comp-subr-trampoline-install(yes-or-no-p)
  {test-overlay-regions} #f(compiled-function () #<bytecode -0x1be13de621723a1f>)()
  test-kill-buffer-auto-save(110 {test-kill-buffer-auto-save} #f(compiled-function () #<bytecode -0x1c782c52a4f77189>))
  {test-kill-buffer-auto-save} #f(compiled-function () #<bytecode 0xeb411732a4fb03f>)()

(The symbols in braces are an enhancement I'm currently working on to give
more information for anonymous functions.)

If anybody can match up the "wrong-type-argument" message to this
backtrace, they're a better hacker than me - What precisely in
comp-spill-lap-function is using some unknown list primitive on the byte
compiled function?

So, on the principle of using ALL the information that is available, we
might as well disassemble that byte-compiled function, which might give a
clue:

    \300   byte-constant yes-or-no-p
    \211   byte-dup
    \2     stack-ref 2 ; ARG124
    !      byte-call 1
    \262\1 stack-set 1
    \207   return

.... but not much of one.  The function is calling yes-or-no-p with its
parameter, and then I think it's returning yes-or-no-p's result.  Though
why it's duplicating yes-or-no-p at the top of the stack, then
overwriting it with the result is unclear.  In the test
test-kill-buffer-auto-save-delete-no, fancy things are done with cl-letf
on (symbol-function 'yes-or-no-p), so the disassembled function above
probably has something to do with that.

It's worth pointing out that there doesn't seem to be a way to get Emacs
to disassemble a function, only a symbol with a function value.

########################################################################

I will eventually track down the cause of the above bug.  But it will
have taken me MUCH longer than it would have done, had the missing
information actually been present in the backtrace.  I think I might not
be alone in wishing for these things to be improved.

So I propose that the quality of our backtraces be improved, and nominate
myself as the person to do the work.

-- 
Alan Mackenzie (Nuremberg, Germany).