backtrace-mode

backtrace-mode

[Gemini Lasswell]

Hello all,

Elisp backtraces are printed in 3 places (subr.el, edebug.el and
debug.el) and displayed in buffers in 3 places (debug.el, edebug.el
and ert.el) without much shared code.  I wanted to improve Edebug's
backtraces, and then to make the same things work in ERT and the
debugger without writing it all 3 times, which got me started writing
a backtrace library.

I've just pushed a new branch scratch/backtrace-mode, containing a new
file backtrace.el with the shared backtrace code and a new
backtrace-mode with better backtrace viewing, along with changes to
ERT, Edebug and the debugger to make them all use it, and supporting
changes in cl-print.el.

This code is a Work In Progress, still liberally sprinkled with TODOs.
But it works and can do nifty stuff like pretty-print backtrace frames
and expand "..."s, so please give it a try and let me know what you
think.  (It's my first major mode and my first time doing anything
with text properties, so there might be some beginner mistakes.)

Questions:

- debugger-stack-frame-as-list now seems like it should be in
  backtrace mode's defcustoms instead of the debugger's.  Should it be
  moved or left where it is?

- Should I move the backtrace related functions from subr.el to
  backtrace.el?  There's a function in backtrace.el currently called
  backtrace-backtrace which could replace subr's backtrace.

- I haven't made a menu yet, but I assume backtrace-mode should have
  one.  Should I use easy-menu-define for that, and why doesn't
  debug.el use easy-menu-define?

- When you enter the debugger, or when you ask ERT to show a
  backtrace, both of them pop to the backtrace buffer.  But when you
  ask Edebug to show a backtrace, it just shows the backtrace buffer.
  I think all three should do the same thing.  Now that Edebug's
  backtrace has commands other than quit, I think it should also be
  popped to.  Any objections?

- Now that xref-find-definitions works in backtrace buffers, I think
  it would be better to stop making buttons for the functions in a
  backtrace, and to document M-. as the best way to jump to function
  definitions from backtraces.  What do you think of this change to
  existing behavior, with or without a new customization variable to
  preserve the old behavior?

- backtrace--print-to-string (which tries to find values of
  print-length and print-level to fit the printed representation of an
  object into a specified number of characters) and ellipsis expansion
  would also be useful for ERT, edebug and debug results printing.  If
  that happens, a way to expand ellipses in *Messages* would be
  nice.  So maybe these features don't belong in backtrace.el,
  but I don't know where they do belong.

- Tab and backtab are used by button navigation.  Seems to me the next
  best choice for bindings for pretty printing and collapsing back to
  one line are '+' and '-' except '-' is bound to negative-argument
  in special-mode.  So I'm currently using '+' and '=', but I don't
  like the '='.  Would it be ok to rebind '-' in backtrace buffers?

Bugs fixed in this branch:

- 18609, fontification in backtraces
- 18558, debugger slow with long strings
- 15749, *Backtrace* loses text scaling
- 1428, links in Edebug backtraces

On my todo list:

- bug#14081, add a link to "reading at buffer pos ..."

- A new command for Edebug backtraces only, 's' to jump to the source
  code for the frame at point.  I'm planning on adding an indicator in
  the left margin (where the regular debugger puts its *) to show
  whether the source is available for a given frame, since the frames
  in a backtrace might be a mix of code instrumented and not
  instrumented by Edebug.


Gemini

Re: backtrace-mode

[Helmut Eller]

On Tue, Jun 19 2018, Gemini Lasswell wrote:

> - Now that xref-find-definitions works in backtrace buffers, I think
>   it would be better to stop making buttons for the functions in a
>   backtrace, and to document M-. as the best way to jump to function
>   definitions from backtraces.  What do you think of this change to
>   existing behavior, with or without a new customization variable to
>   preserve the old behavior?

Apparently M-. in this backtrace-mode uses the elisp backend.  While
that works, a backtrace-frame has potentially more context than just the
name of a global function.  E.g. I have a custom "view source" command
that looks at the next frame and tries to find the function called
inside the function source.  That works pretty well as long as there
aren't too many macros or lambdas involved.  Also the compiler could
potentially emit debug information to map the frame's PC to the source
position; which would be very useful for lambdas.  So I think there
should either be a proper "view source" command or xref backend for
backtraces.

Another issue: backtrace-get-frames seems to compute the entire
backtrace and all possibly relevant information in advance.  Maybe that
should be done a bit more lazily, e.g. compute the next N frames only if
somebody needs them.  For local variable also keep in mind that
backtrace--locals is implemented rather inefficiently.  That would also
be less problematic if locals are only fetched when needed.

Helmut

Re: backtrace-mode

[Gemini Lasswell]

Helmut Eller <eller.helmut@gmail.com> writes:

> So I think there
> should either be a proper "view source" command or xref backend for
> backtraces.

I agree.  I think there should be three commands:

1. "View source", to jump to the source for the frame at point.  I
haven't done it yet but I plan on implementing it and binding it to
the 's' key.  For now it is going to work best with Edebug, but when
the byte compiler or Lisp reader gain the ability someday to save
source code positions, it could be made to work with those as well.

2. "Go to definition", to jump to the source code for the definition
of the symbol at point, which can be used on any function or variable
name in the buffer.  I think xref-find-definitions with the Elisp
backend works well for this.

3. "Show help", to bring up help for the symbol at point, currently
implemented by backtrace-help-follow-symbol, bound to RET.

> Another issue: backtrace-get-frames seems to compute the entire
> backtrace and all possibly relevant information in advance.  Maybe that
> should be done a bit more lazily, e.g. compute the next N frames only if
> somebody needs them.  For local variable also keep in mind that
> backtrace--locals is implemented rather inefficiently.  That would also
> be less problematic if locals are only fetched when needed.

I'm aware that there's an O(n^2) here but I didn't want to prematurely
optimize and make the code more complicated without noticable benefit.
While working on this code I had tried generating backtraces from
max-lisp-eval-depth errors a few times and they always came up
quickly.  Yesterday I realized that was due to a bug, now on debbugs
as #31919.  With that fixed I tried wrapping the calls to
backtrace-get-frames and backtrace-print with benchmark-run, and
then evaluating this code:

(toggle-debug-on-error)
(defun my-func (arg)
  (let ((arg1 arg))
    (+ (length arg) (my-func arg))))

And then I tried both (my-func "hello") and (my-func my-str) with
my-str bound to the buffer-string of a magit-status buffer, which 
for the one I used made (length (cl-prin1-to-string my-str)) evaluate
to 1711302, and which is the sort of value which would make previous
versions of Emacs backtraces excruciatingly slow.

From my experiments I concluded backtrace-get-frames is not a
performance problem.  With standard stack limits it takes about 0.015
seconds, and even after increasing both max-lisp-eval-depth and
max-specpdl-size by a factor of 5, it still only takes 0.2 seconds (on
a 5-year old Intel cpu, 2.6GHz).  But backtrace-print is far slower,
taking 3.7 seconds with high stack limits and "hello" and 60 seconds
with default stack limits and the magit buffer-string.  So I'll look
into what can be done lazily when rendering the backtrace frames into
the buffer.

Gemini

Re: backtrace-mode

[Stefan Monnier]

> 3. "Show help", to bring up help for the symbol at point, currently
> implemented by backtrace-help-follow-symbol, bound to RET.

Doesn't C-h o already do the trick?


        Stefan

Re: backtrace-mode

[Gemini Lasswell]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 3. "Show help", to bring up help for the symbol at point, currently
>> implemented by backtrace-help-follow-symbol, bound to RET.
>
> Doesn't C-h o already do the trick?

Yes it does.  backtrace-help-follow-symbol isn't new, I just renamed
it from debug-help-follow and moved it to backtrace.el so ERT and
Edebug could use it too.  It's mostly just a wrapper around
describe-symbol that skips prompting to confirm the symbol.  It's also
very similar to help-follow-symbol.

Re: backtrace-mode

[Stefan Monnier]

> I've just pushed a new branch scratch/backtrace-mode, containing a new
> file backtrace.el with the shared backtrace code and a new
> backtrace-mode with better backtrace viewing, along with changes to
> ERT, Edebug and the debugger to make them all use it, and supporting
> changes in cl-print.el.

Thanks.  I haven't had a chance to use it yet, nor to really look at its
code, but I saw the commit and I very much like the idea.


        Stefan

Re: backtrace-mode

[Filipp Gunbin]

On 19/06/2018 15:58 -0700, Gemini Lasswell wrote:

> - Now that xref-find-definitions works in backtrace buffers, I think
>   it would be better to stop making buttons for the functions in a
>   backtrace, and to document M-. as the best way to jump to function
>   definitions from backtraces.  What do you think of this change to
>   existing behavior, with or without a new customization variable to
>   preserve the old behavior?

I'd rather have buttons as well.

> - Tab and backtab are used by button navigation.  Seems to me the next
>   best choice for bindings for pretty printing and collapsing back to
>   one line are '+' and '-' except '-' is bound to negative-argument
>   in special-mode.  So I'm currently using '+' and '=', but I don't
>   like the '='.  Would it be ok to rebind '-' in backtrace buffers?

Maybe something with `-e' for expand and `-c' for collapse, like
`C-c'/`C-e' in outline-mode?

Filipp