bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

I wrote a minimal patch that increases the overall consistency in a
backtrace buffer by printing the call stack frames as S-Expressions.

Before:

Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
  +(1 t)
  eval((+ 1 t) nil)
  eval-expression((+ 1 t) nil)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)

After:

Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
  (debug error (wrong-type-argument number-or-marker-p t))
  (+ 1 t)
  (eval (+ 1 t) nil)
  (eval-expression (+ 1 t) nil)
  (funcall-interactively eval-expression (+ 1 t) nil)
  (call-interactively eval-expression nil nil)
  (command-execute eval-expression)

Now, this patch isn't perfect.  For some reason there's an extra debug
line in the second version, I've yet to investigate into the reason for
this.  The other problem is that while I can't imagine any reason to go
back to the original view of the backtrace, I cannot rule out that this
change might break other tools relying on it.  I'd appreciate any
feedback on this.

[-- Attachment #2: 0001-Make-backtraces-great-again.patch --]
[-- Type: text/x-diff, Size: 1342 bytes --]

From 232cb613a83f128d8ee90d7a52fcbde06fd29766 Mon Sep 17 00:00:00 2001
From: Vasilij Schneidermann <v.schneidermann@gmail.com>
Date: Thu, 22 Sep 2016 23:01:21 +0200
Subject: [PATCH] Make backtraces great again

---
 lisp/emacs-lisp/debug.el | 2 +-
 src/eval.c               | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 22a3f39..4020620 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -279,7 +279,7 @@ That buffer should be current already."
   (goto-char (point-min))
   (delete-region (point)
 		 (progn
-		   (search-forward "\n  debug(")
+		   (search-forward "\n  (debug")
 		   (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
diff --git a/src/eval.c b/src/eval.c
index 72facd5..e32e7a1 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3409,8 +3409,9 @@ Output stream used is value of `standard-output'.  */)
       else
 	{
 	  tem = backtrace_function (pdl);
-	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
 	  write_string ("(");
+	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
+	  write_string (" ");
 	  {
 	    ptrdiff_t i;
 	    for (i = 0; i < backtrace_nargs (pdl); i++)
-- 
2.9.3

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1706 bytes --]

This looks great! I love it.  And the patch looks very clean, too.

But it scares me a bit.  Some tools do depend on e.g. trimming a backtrace after printing it.  Does edebug work with your patch, for example?

I'm not sure what the right way to transition is.  Maybe Emacs should let Lisp programs access the backtraces in a structured way, and then backtrace printing would only be a user-facing facility (programs wouldn't use the textual representation).

Cheers,
Clément.

On 2016-09-22 19:14, Vasilij Schneidermann wrote:
> I wrote a minimal patch that increases the overall consistency in a
> backtrace buffer by printing the call stack frames as S-Expressions.
> 
> Before:
> 
> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
>   +(1 t)
>   eval((+ 1 t) nil)
>   eval-expression((+ 1 t) nil)
>   call-interactively(eval-expression nil nil)
>   command-execute(eval-expression)
> 
> After:
> 
> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
>   (debug error (wrong-type-argument number-or-marker-p t))
>   (+ 1 t)
>   (eval (+ 1 t) nil)
>   (eval-expression (+ 1 t) nil)
>   (funcall-interactively eval-expression (+ 1 t) nil)
>   (call-interactively eval-expression nil nil)
>   (command-execute eval-expression)
> 
> Now, this patch isn't perfect.  For some reason there's an extra debug
> line in the second version, I've yet to investigate into the reason for
> this.  The other problem is that while I can't imagine any reason to go
> back to the original view of the backtrace, I cannot rule out that this
> change might break other tools relying on it.  I'd appreciate any
> feedback on this.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Vasilij Schneidermann]

> This looks great! I love it.  And the patch looks very clean, too.

Thanks!  I've updated the patch after a bit more testing.  The extra
debug line happened because the search in debug.el did actually search
for a `debug' call, so `(search-forward "\  (debug")` would match a call
to `debug-foo` as well.  Assuming there will always be extra args, this
can be solved by appending a space to the search string.  In case this
is not true, I'll have to go for `re-search-forward` to handle a
`(debug)` as well.  The other problem was that there were extraneous
spaces at times, this can be handled by making the line that prepends an
argument with a space unconditional.

> But it scares me a bit.  Some tools do depend on e.g. trimming a
> backtrace after printing it.  Does edebug work with your patch, for
> example?

Yes, it does.  However, you're raising a good point here.  I would
expect such a change to neither break Emacs core nor any of the bundled
Emacs Lisp code.  Guarantees about popular external code are hard to
make, but communicating the change with the respective authors and
maintainers should do the trick.  What I've initially asked for is
whether there might be any good reason for *not* doing things this way.

I've grepped the 24.5 sources and most usage of `backtrace' appears to
be of diagnostic nature.  The only thing other than `debug.el`
manipulating them is `edebug-backtrace' which I've never heard of
before.  I'll take a better look at its sources for my next revision of
the patch.

> I'm not sure what the right way to transition is.  Maybe Emacs should
> let Lisp programs access the backtraces in a structured way, and then
> backtrace printing would only be a user-facing facility (programs
> wouldn't use the textual representation).

There is actually a way to do this, simply use `backtrace-frame' with an
increasing integer argument until it returns nil:

    (let ((frame t) ; dummy value to kick-start the loop
          (i 0))
      (while frame
        (setq frame (backtrace-frame i))
        (message "%S" frame)
        (setq i (1+ i))))

This feature is used in `ert.el` to save a backtrace and reconstruct it.
Perhaps all current users of `backtrace' that rely on its current
representation should be using `backtrace-frame' instead?

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1782 bytes --]

On 2016-09-23 03:51, Vasilij Schneidermann wrote:
>> This looks great! I love it.  And the patch looks very clean, too.
> 
> Thanks!  I've updated the patch after a bit more testing.  The extra 
> debug line happened because the search in debug.el did actually
> search for a `debug' call, so `(search-forward "\  (debug")` would
> match a call to `debug-foo` as well. Assuming there will always be
> extra args, this can be solved by appending a space to the search
> string.

Could it just search for the first occurence only?

>> But it scares me a bit.  Some tools do depend on e.g. trimming a 
>> backtrace after printing it.  Does edebug work with your patch,
>> for example?
> 
> Yes, it does.  …  The only thing other than `debug.el` 
> manipulating them is `edebug-backtrace' which I've never heard of 
> before. 

This is the one I was scared about. You can get a backtrace while edebugging by pressing t IIRC.  I have a faint memory of this code doing nasty things to remove edebug instrumentation from the backtrace before displaying it.

>> I'm not sure what the right way to transition is.  Maybe Emacs
>> should let Lisp programs access the backtraces in a structured way,
>> and then backtrace printing would only be a user-facing facility
>> (programs wouldn't use the textual representation).
> 
> There is actually a way to do this, simply use `backtrace-frame' with
> an increasing integer argument until it returns nil:
> 
> (let ((frame t) ; dummy value to kick-start the loop (i 0)) (while
> frame (setq frame (backtrace-frame i)) (message "%S" frame) (setq i
> (1+ i))))

Neat! Didn't know about this.  I think it would be great to add a pointer to this in the documentation of `backtrace'.

Great work :)
Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: bug#24514: 24.5; Lispy backtraces

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 2122 bytes --]

Hi emacs-devel,

The discussion below seems to indicate that there's very little missing at the Lisp level to be able to implement `backtrace' in Lisp; is that right?

The documentation for backtrace states:

     It is written in C, since it must have
     access to the stack to determine which function calls are active.
     The return value is always ‘nil’.

But the existence of backtrace-frame seems to (partially) contradict this; or am I misunderstanding?

The context is that I need to capture backtraces on an Emacs server, and that I have no way (AFAICT) to know, when my debugger gets invoked, whether the error will then be handled by a condition-case block; thus I end up recording too much and my program ends up spending 95% of its time capturing useless backtraces.  

Instead, recording a sequence of backtrace-frame seems fast (thanks Vasilij!).  But then, if an unhandled exception occurs, I need to actually format the recorded stack frames; which seems to be what `backtrace' does, only at the C level.

Cheers
Clément. 

On 2016-12-01 19:52, Vasilij Schneidermann wrote:
>> Thinking more about this, isn't this enough to implement `backtrace' in Lisp?
> 
> Not quite.  If you look at backtraces, you'll notice they are constantly
> prefixed with two spaces.  This prefix is conditional and can be
> replaced with an asterisk and a space to indicate where you currently
> are when stepping through the backtrace.  From what I can tell, there is
> no way to retrieve that information when relying on `backtrace-frame'
> only.
> 
> If you ignore that part, it's not too hard to write a replacement:
> 
> (with-output-to-string
>   (let ((print-level (or print-level 8))
>         (i 0)
>         frame)
>     (while (setq frame (backtrace-frame i))
>       (princ "  ")
>       (if (not (car frame))
>           (prin1 (cdr frame))
>         (prin1 (cadr frame))
>         (prin1 (cddr frame)))
>       (terpri)
>       (setq i (1+ i)))))
> 
> How faithful that one is, I don't know.  Perhaps this is a discussion better
> held on emacs-devel?
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: bug#24514: 24.5; Lispy backtraces

[Stefan Monnier]

> The discussion below seems to indicate that there's very little missing at
> the Lisp level to be able to implement `backtrace' in Lisp; is that right?

Indeed, I think all the info needed is provided by backtrace-frame.


        Stefan

Re: bug#24514: 24.5; Lispy backtraces

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 565 bytes --]

On 2016-12-01 21:24, Stefan Monnier wrote:
>> The discussion below seems to indicate that there's very little missing at
>> the Lisp level to be able to implement `backtrace' in Lisp; is that right?
> 
> Indeed, I think all the info needed is provided by backtrace-frame.

The C implementation of backtrace-frame seems to be linear in the index of the requested frame, so a Lisp implementation of backtrace would be quadratic in the depth of the stack trace.  Would a new function backtrace-frames that returns all frames at once be acceptable?

Clément


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: bug#24514: 24.5; Lispy backtraces

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Sat, 3 Dec 2016 17:15:22 -0500
> 
> On 2016-12-01 21:24, Stefan Monnier wrote:
> >> The discussion below seems to indicate that there's very little missing at
> >> the Lisp level to be able to implement `backtrace' in Lisp; is that right?
> > 
> > Indeed, I think all the info needed is provided by backtrace-frame.
> 
> The C implementation of backtrace-frame seems to be linear in the index of the requested frame, so a Lisp implementation of backtrace would be quadratic in the depth of the stack trace.  Would a new function backtrace-frames that returns all frames at once be acceptable?

But such a backtrace-frames function would have to be implemented in
C, right?  And you wanted to move the implementation of "backtrace" to
Lisp, AFAIU.  So it sounds like we will be replacing one C primitive
with another, or did I miss something?

Re: bug#24514: 24.5; Lispy backtraces

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1264 bytes --]

On 2016-12-04 10:30, Eli Zaretskii wrote:
>> From: Clément Pit--Claudel <clement.pit@gmail.com>
>> Date: Sat, 3 Dec 2016 17:15:22 -0500
>>
>> On 2016-12-01 21:24, Stefan Monnier wrote:
>>>> The discussion below seems to indicate that there's very little missing at
>>>> the Lisp level to be able to implement `backtrace' in Lisp; is that right?
>>>
>>> Indeed, I think all the info needed is provided by backtrace-frame.
>>
>> The C implementation of backtrace-frame seems to be linear in the index of the requested frame, so a Lisp implementation of backtrace would be quadratic in the depth of the stack trace.  Would a new function backtrace-frames that returns all frames at once be acceptable?
> 
> But such a backtrace-frames function would have to be implemented in
> C, right?  And you wanted to move the implementation of "backtrace" to
> Lisp, AFAIU.  So it sounds like we will be replacing one C primitive
> with another, or did I miss something?

I think you're correct. It would seem good to have the flexible primitive backtrace-frames available, and it must be in C; then we can move backtrace itself to lisp.

The idea is that enumerating frames must be done in C, but printing them doesn't need to be done there.

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: bug#24514: 24.5; Lispy backtraces

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Sun, 4 Dec 2016 14:27:59 -0500
> 
> >> The C implementation of backtrace-frame seems to be linear in the index of the requested frame, so a Lisp implementation of backtrace would be quadratic in the depth of the stack trace.  Would a new function backtrace-frames that returns all frames at once be acceptable?
> > 
> > But such a backtrace-frames function would have to be implemented in
> > C, right?  And you wanted to move the implementation of "backtrace" to
> > Lisp, AFAIU.  So it sounds like we will be replacing one C primitive
> > with another, or did I miss something?
> 
> I think you're correct. It would seem good to have the flexible primitive backtrace-frames available, and it must be in C; then we can move backtrace itself to lisp.
> 
> The idea is that enumerating frames must be done in C, but printing them doesn't need to be done there.

So would it perhaps make sense to rename 'backtrace' into something
like 'backtrace--internal', and make it accept one more argument, the
function to apply to each frame, which is now hard-coded as 'prin1'?
Would that allow you to implement 'backtrace' in Lisp and also
implement whatever application you had in mind, by calling
'backtrace--internal' passing it your own function instead of 'prin1'?

Re: bug#24514: 24.5; Lispy backtraces

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1931 bytes --]

On 2016-12-04 15:41, Eli Zaretskii wrote:
>> Cc: emacs-devel@gnu.org
>> From: Clément Pit--Claudel <clement.pit@gmail.com>
>> Date: Sun, 4 Dec 2016 14:27:59 -0500
>>
>>>> The C implementation of backtrace-frame seems to be linear in the index of the requested frame, so a Lisp implementation of backtrace would be quadratic in the depth of the stack trace.  Would a new function backtrace-frames that returns all frames at once be acceptable?
>>>
>>> But such a backtrace-frames function would have to be implemented in
>>> C, right?  And you wanted to move the implementation of "backtrace" to
>>> Lisp, AFAIU.  So it sounds like we will be replacing one C primitive
>>> with another, or did I miss something?
>>
>> I think you're correct. It would seem good to have the flexible primitive backtrace-frames available, and it must be in C; then we can move backtrace itself to lisp.
>>
>> The idea is that enumerating frames must be done in C, but printing them doesn't need to be done there.
> 
> So would it perhaps make sense to rename 'backtrace' into something
> like 'backtrace--internal', and make it accept one more argument, the
> function to apply to each frame, which is now hard-coded as 'prin1'?
> Would that allow you to implement 'backtrace' in Lisp and also
> implement whatever application you had in mind, by calling
> 'backtrace--internal' passing it your own function instead of 'prin1'?

Quite possibly! Are you worried about the cost of allocating a list containing all frames? 
IN any case, that would be consistent with the style of maphash, for example. And a variant of backtrace taking a callback would definitely make backtrace-frames easy to implement on the lisp side :)

There's no big hurry on this, anyway: it's just that we recently added a neat option to backtrace, and there were mentions of making the backtrace buffer more useful in various ways, too.

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: bug#24514: 24.5; Lispy backtraces

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Sun, 4 Dec 2016 17:14:38 -0500
> 
> > So would it perhaps make sense to rename 'backtrace' into something
> > like 'backtrace--internal', and make it accept one more argument, the
> > function to apply to each frame, which is now hard-coded as 'prin1'?
> > Would that allow you to implement 'backtrace' in Lisp and also
> > implement whatever application you had in mind, by calling
> > 'backtrace--internal' passing it your own function instead of 'prin1'?
> 
> Quite possibly! Are you worried about the cost of allocating a list containing all frames? 

No, I'm not worried about that.

> a variant of backtrace taking a callback would definitely make backtrace-frames easy to implement on the lisp side :)

Yup.

> There's no big hurry on this, anyway: it's just that we recently added a neat option to backtrace, and there were mentions of making the backtrace buffer more useful in various ways, too.

Will you be working on this idea at some point?

Lisp-friendly backtraces [was: Lispy backtraces]

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 1035 bytes --]

On 2016-12-04 22:30, Eli Zaretskii wrote:
>>> So would it perhaps make sense to rename 'backtrace' into something
>>> like 'backtrace--internal', and make it accept one more argument, the
>>> function to apply to each frame, which is now hard-coded as 'prin1'?
>>> Would that allow you to implement 'backtrace' in Lisp and also
>>> implement whatever application you had in mind, by calling
>>> 'backtrace--internal' passing it your own function instead of 'prin1'?
>>
>> Quite possibly!
>
> Will you be working on this idea at some point?

Yup.  I've attached a rough patch, and an ELisp file showing what calling the new function (mapbacktrace) looks like.

Let me know if this direction makes sense.  If so, I will write the corresponding documentation, a Changelog entry, and a proper commit message.

(CC Alan, since you recently expressed frustration with the contents of the *Backtrace* buffer; hopefully this new function will help with implementing a more flexible *Backtrace* buffer?)

Cheers,
Clément.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-New-function-mapbacktrace.patch --]
[-- Type: text/x-diff; name="0001-New-function-mapbacktrace.patch", Size: 2753 bytes --]

From 861c841eb242c474c66421ce4d9964940033ff31 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Mon, 5 Dec 2016 00:52:14 -0500
Subject: [PATCH] New function mapbacktrace

---
 src/eval.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)

diff --git a/src/eval.c b/src/eval.c
index 724f001..dcda51c 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3420,6 +3420,53 @@ The debugger is entered when that frame exits, if the flag is non-nil.  */)
   return flag;
 }
 
+DEFUN ("mapbacktrace", Fmapbacktrace, Smapbacktrace, 1, 2, 0,
+       doc: /* Call FUNCTION for each frame in backtrace.
+FUNCTION is called with 4 arguments EVALD FUNC ARGS FLAGS.  If a frame
+has not evaluated its arguments yet or is a special form, EVALD is nil
+and ARGS is a list of forms.  If a frame has evaluated its arguments
+and called its function already, EVALD is t and ARGS is a list of
+values.  FLAGS is a plist of properties of the current frame:
+currently, the only supported property is :debug-on-exit.
+If NSKIP is non-nil, the top NSKIP frames are skipped.
+`mapbacktrace' always returns nil.  */)
+     (Lisp_Object function, Lisp_Object nskip)
+{
+  union specbinding *pdl = backtrace_top ();
+
+  if (!NILP (nskip))
+    {
+      CHECK_NUMBER(nskip);
+      EMACS_INT to_skip = XINT(nskip);
+      while (to_skip > 0 && backtrace_p (pdl)) {
+        to_skip--;
+        pdl = backtrace_next (pdl);
+      }
+    }
+
+  while (backtrace_p (pdl))
+    {
+      Lisp_Object flags = Qnil;
+      if (backtrace_debug_on_exit (pdl))
+        {
+          flags = Fcons (QCdebug_on_exit, Fcons (Qt, Qnil));
+        }
+
+      if (backtrace_nargs (pdl) == UNEVALLED)
+        {
+          call4 (function, Qnil, backtrace_function (pdl), *backtrace_args (pdl), flags);
+        }
+      else
+        {
+          Lisp_Object tem = Flist (backtrace_nargs (pdl), backtrace_args (pdl));
+          call4 (function, Qt, backtrace_function (pdl), tem, flags);
+        }
+      pdl = backtrace_next (pdl);
+    }
+
+  return Qnil;
+}
+
 DEFUN ("backtrace", Fbacktrace, Sbacktrace, 0, 0, "",
        doc: /* Print a trace of Lisp function calls currently active.
 Output stream used is value of `standard-output'.  */)
@@ -3973,7 +4020,8 @@ alist of active lexical bindings.  */);
   defsubr (&Srun_hook_wrapped);
   defsubr (&Sfetch_bytecode);
   defsubr (&Sbacktrace_debug);
-  defsubr (&Sbacktrace);
+  DEFSYM (QCdebug_on_exit, ":debug-on-exit");
+  defsubr (&Smapbacktrace);
   defsubr (&Sbacktrace_frame);
   defsubr (&Sbacktrace_eval);
   defsubr (&Sbacktrace__locals);
-- 
2.7.4


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.3: bt.el --]
[-- Type: text/x-emacs-lisp; name="bt.el", Size: 1931 bytes --]

;; -*- lexical-binding: t -*-

(defun backtrace-1 (evald func args flags)
  "Print a trace of a single stack frame to `standard-output'.
EVALD, FUNC, ARGS, FLAGS are as in `mapbacktrace'."
  (let ((print-level (or print-level 8)))
    (princ (if (plist-get flags :debug-on-exit) "* " "  "))
    (cond
     ((and evald (not debugger-stack-frame-as-list))
      (prin1 func)
      (if args (prin1 args) (princ "()")))
     (t
      (prin1 (cons func args))))
    (princ "\n")))

(defun backtrace ()
  "Print a trace of Lisp function calls currently active.
Output stream used is value of `standard-output'."
  (mapbacktrace #'~/backtrace-1 1))

(defun backtrace-frames ()
  "Collect all frames of current backtrace into a list."
  (let ((frames nil))
    (mapbacktrace (lambda (&rest frame) (push frame frames)) 2)
    (nreverse frames)))

(defun ~/backtrace-frame (nframes &optional base)
  "Return the function and arguments NFRAMES up from current execution point.
If that frame has not evaluated the arguments yet (or is a special form),
the value is (nil FUNCTION ARG-FORMS...).
If that frame has evaluated its arguments and called its function already,
the value is (t FUNCTION ARG-VALUES...).
A &rest arg is represented as the tail of the list ARG-VALUES.
FUNCTION is whatever was supplied as car of evaluated list,
or a lambda expression for macro calls.
If NFRAMES is more than the number of frames, the value is nil.
If BASE is non-nil, it should be a function and NFRAMES counts from its
nearest activation frame."
  (let ((frame nil))
    (mapbacktrace (lambda (evald func args _)
                    (when (and base (eq func base))
                      (setq base nil))
                    (unless base
                      (when (eq nframes 0)
                        (setq frame `(,evald ,func ,@args)))
                      (setq nframes (1- nframes)))))
    frame))

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Stefan Monnier]

> +  if (!NILP (nskip))
> +    {
> +      CHECK_NUMBER(nskip);
                    ^^
Please put a space before every open paren.

> +      EMACS_INT to_skip = XINT(nskip);
> +      while (to_skip > 0 && backtrace_p (pdl)) {
> +        to_skip--;
> +        pdl = backtrace_next (pdl);
> +      }
> +    }

Why not use the same `base` arg as `backtrace-frame` instead of `nskip`?

> -  defsubr (&Sbacktrace);
> +  DEFSYM (QCdebug_on_exit, ":debug-on-exit");
> +  defsubr (&Smapbacktrace);

You remove the defsubr of Sbacktrace, but you don't remove the
corresponding DEFUN.

> (defun backtrace ()
>   "Print a trace of Lisp function calls currently active.
> Output stream used is value of `standard-output'."
>   (mapbacktrace #'~/backtrace-1 1))

Have you tried it both byte-compiled and interpreted?  Maybe this
function is just simple enough that the result is the same in both
cases, but in my experience, the stack is sufficiently different in the
two cases that a constant nskip doesn't cut it (hence the use of `base`
in backtrace-frame).


        Stefan

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 692 bytes --]

On 2016-12-05 08:20, Stefan Monnier wrote:
>> (defun backtrace ()
>>   "Print a trace of Lisp function calls currently active.
>> Output stream used is value of `standard-output'."
>>   (mapbacktrace #'~/backtrace-1 1))
> 
> Have you tried it both byte-compiled and interpreted?  Maybe this
> function is just simple enough that the result is the same in both
> cases, but in my experience, the stack is sufficiently different in the
> two cases that a constant nskip doesn't cut it (hence the use of `base`
> in backtrace-frame).

Thanks; I attached an updated patch.  Removing `backtrace' from eval.c makes the patch much harder to read, so I'll do that later.

Clément.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: bt.el --]
[-- Type: text/x-emacs-lisp; name="bt.el", Size: 2029 bytes --]

;; -*- lexical-binding: t -*-

(defun backtrace-1 (evald func args flags)
  "Print a trace of a single stack frame to `standard-output'.
EVALD, FUNC, ARGS, FLAGS are as in `mapbacktrace'."
  (let ((print-level (or print-level 8)))
    (princ (if (plist-get flags :debug-on-exit) "* " "  "))
    (cond
     ((and evald (not debugger-stack-frame-as-list))
      (prin1 func)
      (if args (prin1 args) (princ "()")))
     (t
      (prin1 (cons func args))))
    (princ "\n")))

(defun backtrace ()
  "Print a trace of Lisp function calls currently active.
Output stream used is value of `standard-output'."
  (mapbacktrace #'backtrace-1 'backtrace))

(backtrace)

(defun backtrace-frames ()
  "Collect all frames of current backtrace into a list."
  (let ((frames nil))
    (mapbacktrace (lambda (&rest frame) (push frame frames)) 'backtrace-frames)
    (nreverse frames)))

(backtrace-frames)

(defun ~/backtrace-frame (nframes &optional base)
  "Return the function and arguments NFRAMES up from current execution point.
If that frame has not evaluated the arguments yet (or is a special form),
the value is (nil FUNCTION ARG-FORMS...).
If that frame has evaluated its arguments and called its function already,
the value is (t FUNCTION ARG-VALUES...).
A &rest arg is represented as the tail of the list ARG-VALUES.
FUNCTION is whatever was supplied as car of evaluated list,
or a lambda expression for macro calls.
If NFRAMES is more than the number of frames, the value is nil.
If BASE is non-nil, it should be a function and NFRAMES counts from its
nearest activation frame."
  (let ((frame nil))
    (mapbacktrace (lambda (evald func args _)
                    (when (and base (eq func base))
                      (setq base nil))
                    (unless base
                      (when (eq nframes 0)
                        (setq frame `(,evald ,func ,@args)))
                      (setq nframes (1- nframes))))
                  '~/backtrace-frame)
    frame))

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.3: 0001-New-function-mapbacktrace.patch --]
[-- Type: text/x-diff; name="0001-New-function-mapbacktrace.patch", Size: 3683 bytes --]

From 6302757b6fc664a8ef56ff8742aaf1987e58107d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Mon, 5 Dec 2016 00:52:14 -0500
Subject: [PATCH] New function mapbacktrace

---
 src/eval.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 57 insertions(+), 9 deletions(-)

diff --git a/src/eval.c b/src/eval.c
index 724f001..66b665e 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3420,6 +3420,60 @@ The debugger is entered when that frame exits, if the flag is non-nil.  */)
   return flag;
 }
 
+static union specbinding *
+get_backtrace_starting_at (Lisp_Object base)
+{
+  union specbinding *pdl = backtrace_top ();
+
+  if (!NILP (base))
+    { /* Skip up to `base'.  */
+      base = Findirect_function (base, Qt);
+      while (backtrace_p (pdl)
+             && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
+        pdl = backtrace_next (pdl);
+    }
+
+  return pdl;
+}
+
+DEFUN ("mapbacktrace", Fmapbacktrace, Smapbacktrace, 1, 2, 0,
+       doc: /* Call FUNCTION for each frame in backtrace.
+FUNCTION is called with 4 arguments EVALD FUNC ARGS FLAGS.  If a frame
+has not evaluated its arguments yet or is a special form, EVALD is nil
+and ARGS is a list of forms.  If a frame has evaluated its arguments
+and called its function already, EVALD is t and ARGS is a list of
+values.  FLAGS is a plist of properties of the current frame:
+currently, the only supported property is :debug-on-exit.
+If BASE is non-nil, it should be a function and iteration will start
+from its nearest activation frame.
+`mapbacktrace' always returns nil.  */)
+     (Lisp_Object function, Lisp_Object base)
+{
+  union specbinding *pdl = get_backtrace_starting_at (base);
+
+  while (backtrace_p (pdl))
+    {
+      Lisp_Object flags = Qnil;
+      if (backtrace_debug_on_exit (pdl))
+        {
+          flags = Fcons (QCdebug_on_exit, Fcons (Qt, Qnil));
+        }
+
+      if (backtrace_nargs (pdl) == UNEVALLED)
+        {
+          call4 (function, Qnil, backtrace_function (pdl), *backtrace_args (pdl), flags);
+        }
+      else
+        {
+          Lisp_Object tem = Flist (backtrace_nargs (pdl), backtrace_args (pdl));
+          call4 (function, Qt, backtrace_function (pdl), tem, flags);
+        }
+      pdl = backtrace_next (pdl);
+    }
+
+  return Qnil;
+}
+
 DEFUN ("backtrace", Fbacktrace, Sbacktrace, 0, 0, "",
        doc: /* Print a trace of Lisp function calls currently active.
 Output stream used is value of `standard-output'.  */)
@@ -3470,18 +3524,10 @@ Output stream used is value of `standard-output'.  */)
 static union specbinding *
 get_backtrace_frame (Lisp_Object nframes, Lisp_Object base)
 {
-  union specbinding *pdl = backtrace_top ();
   register EMACS_INT i;
 
   CHECK_NATNUM (nframes);
-
-  if (!NILP (base))
-    { /* Skip up to `base'.  */
-      base = Findirect_function (base, Qt);
-      while (backtrace_p (pdl)
-	     && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
-	pdl = backtrace_next (pdl);
-    }
+  union specbinding *pdl = get_backtrace_starting_at (base);
 
   /* Find the frame requested.  */
   for (i = XFASTINT (nframes); i > 0 && backtrace_p (pdl); i--)
@@ -3974,6 +4020,8 @@ alist of active lexical bindings.  */);
   defsubr (&Sfetch_bytecode);
   defsubr (&Sbacktrace_debug);
   defsubr (&Sbacktrace);
+  DEFSYM (QCdebug_on_exit, ":debug-on-exit");
+  defsubr (&Smapbacktrace);
   defsubr (&Sbacktrace_frame);
   defsubr (&Sbacktrace_eval);
   defsubr (&Sbacktrace__locals);
-- 
2.7.4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Stefan Monnier]

> Removing `backtrace' from eval.c makes the patch much harder to read,
> so I'll do that later.

M-x diff-unified->context RET

should solve this apparent problem.


        Stefan

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 680 bytes --]

On 2016-12-05 09:37, Stefan Monnier wrote:
>> Removing `backtrace' from eval.c makes the patch much harder to read,
>> so I'll do that later.
> 
> M-x diff-unified->context RET
> 
> should solve this apparent problem.

Neat!  I've attached a cleaner patch, including documentation and a Changelog entry.

Help with the following warning would be much appreciated:

  eval.c:3436:1: warning: no previous prototype for ‘backtrace_frame_apply’ [-Wmissing-prototypes]
   backtrace_frame_apply (Lisp_Object function, union specbinding *pdl)
   ^

(why does this specific function cause this warning, while other newly introduced functions don't?)

Thanks,
Clément.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-Move-backtrace-to-ELisp-using-a-new-mapbacktrace-pri.patch --]
[-- Type: text/x-diff; name="0001-Move-backtrace-to-ELisp-using-a-new-mapbacktrace-pri.patch", Size: 13696 bytes --]

From 185736cd23be13677ef3528cf83ef4a4f3039c72 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Mon, 5 Dec 2016 00:52:14 -0500
Subject: [PATCH] Move backtrace to ELisp using a new mapbacktrace primitive

* src/eval.c (get_backtrace_starting_at, backtrace_frame_apply)
(Fmapbacktrace, Fbacktrace_frame_internal): New functions.
(get_backtrace_frame, Fbacktrace_debug): Use `get_backtrace_starting_at'.

* lisp/subr.el (backtrace--print-frame): New function.
(backtrace): Reimplement using `backtrace--print-frame' and `mapbacktrace'.
(backtrace-frame): Reimplement using `backtrace-frame--internal'.

* lisp/emacs-lisp/debug.el (debugger-setup-buffer): Pass a
base to `mapbacktrace' instead of searching for "(debug" in
the output of `backtrace'.

* doc/lispref/debugging.texi (Internals of Debugger): Document
`mapbacktrace' and missing argument BASE of `backtrace-frame'.
---
 doc/lispref/debugging.texi |  24 ++++++-
 etc/NEWS                   |   4 ++
 lisp/emacs-lisp/debug.el   |  11 ++--
 lisp/subr.el               |  36 ++++++++++
 src/eval.c                 | 160 ++++++++++++++++++++-------------------------
 5 files changed, 140 insertions(+), 95 deletions(-)

diff --git a/doc/lispref/debugging.texi b/doc/lispref/debugging.texi
index c80b0f9..35ff9af 100644
--- a/doc/lispref/debugging.texi
+++ b/doc/lispref/debugging.texi
@@ -727,7 +727,7 @@ Internals of Debugger
 This variable is obsolete and will be removed in future versions.
 @end defvar
 
-@defun backtrace-frame frame-number
+@defun backtrace-frame frame-number &optional base
 The function @code{backtrace-frame} is intended for use in Lisp
 debuggers.  It returns information about what computation is happening
 in the stack frame @var{frame-number} levels down.
@@ -744,10 +744,32 @@ Internals of Debugger
 case of a macro call.  If the function has a @code{&rest} argument, that
 is represented as the tail of the list @var{arg-values}.
 
+If @var{base} is specified, @var{frame-number} counts relative to
+the topmost frame whose @var{function} is @var{base}.
+
 If @var{frame-number} is out of range, @code{backtrace-frame} returns
 @code{nil}.
 @end defun
 
+@defun mapbacktrace function &optional base
+The function @code{mapbacktrace} calls @var{function} once for each
+frame in the backtrace, starting at the first frame whose
+@var{function} is base (or from the top if @var{base} is omitted or
+@code{nil}).
+
+@var{function} is called with four arguments @var{evald} @var{func}
+@var{args} @var{flags}.
+
+If a frame has not evaluated its arguments yet or is a special form,
+@var{evald} is @code{nil} and @var{args} is a list of forms.
+
+If a frame has evaluated its arguments and called its function
+already, @var{evald} is @code{t} and @var{args} is a list of values.
+@var{flags} is a plist of properties of the current frame: currently,
+the only supported property is @code{:debug-on-exit}, which is t if
+the frame's debug-on-exit flag is set.
+@end defun
+
 @include edebug.texi
 
 @node Syntax Errors
diff --git a/etc/NEWS b/etc/NEWS
index a62668a..72bef06 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -74,6 +74,10 @@ for '--daemon'.
 * Changes in Emacs 26.1
 
 +++
+** The new function 'mapbacktrace' applies a function to all frames of
+the current stack trace.
+
++++
 ** The new function 'file-name-case-insensitive-p' tests whether a
 given file is on a case-insensitive filesystem.
 
diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 5430b72..5a4b097 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -274,15 +274,14 @@ debugger-setup-buffer
   (let ((standard-output (current-buffer))
 	(print-escape-newlines t)
 	(print-level 8)
-	(print-length 50))
-    (backtrace))
+        (print-length 50))
+    ;; FIXME the debugger could pass a custom callback to mapbacktrace
+    ;; instead of manipulating printed results.
+    (mapbacktrace #'backtrace--print-frame 'debug))
   (goto-char (point-min))
   (delete-region (point)
 		 (progn
-		   (search-forward (if debugger-stack-frame-as-list
-                                       "\n  (debug "
-                                     "\n  debug("))
-		   (forward-line (if (eq (car args) 'debug)
+                   (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
 				     3
diff --git a/lisp/subr.el b/lisp/subr.el
index 5da5bf8..cb32e66 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -4333,6 +4333,42 @@ define-mail-user-agent
   (put symbol 'sendfunc sendfunc)
   (put symbol 'abortfunc (or abortfunc 'kill-buffer))
   (put symbol 'hookvar (or hookvar 'mail-send-hook)))
+
+\f
+(defun backtrace--print-frame (evald func args flags)
+  "Print a trace of a single stack frame to `standard-output'.
+EVALD, FUNC, ARGS, FLAGS are as in `mapbacktrace'."
+  (princ (if (plist-get flags :debug-on-exit) "* " "  "))
+  (cond
+   ((and evald (not debugger-stack-frame-as-list))
+    (prin1 func)
+    (if args (prin1 args) (princ "()")))
+   (t
+    (prin1 (cons func args))))
+  (princ "\n"))
+
+(defun backtrace ()
+  "Print a trace of Lisp function calls currently active.
+Output stream used is value of `standard-output'."
+  (let ((print-level (or print-level 8)))
+    (mapbacktrace #'backtrace--print-frame 'backtrace)))
+
+(defun backtrace-frame (nframes &optional base)
+  "Return the function and arguments NFRAMES up from current execution point.
+If that frame has not evaluated the arguments yet (or is a special form),
+the value is (nil FUNCTION ARG-FORMS...).
+If that frame has evaluated its arguments and called its function already,
+the value is (t FUNCTION ARG-VALUES...).
+A &rest arg is represented as the tail of the list ARG-VALUES.
+FUNCTION is whatever was supplied as car of evaluated list,
+or a lambda expression for macro calls.
+If NFRAMES is more than the number of frames, the value is nil.
+If BASE is non-nil, it should be a function and NFRAMES counts from its
+nearest activation frame."
+  (backtrace-frame--internal
+   (lambda (evald func args _) `(,evald ,func ,@args))
+   nframes (or base 'backtrace-frame)))
+
 \f
 (defvar called-interactively-p-functions nil
   "Special hook called to skip special frames in `called-interactively-p'.
diff --git a/src/eval.c b/src/eval.c
index 724f001..9baa811 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3401,87 +3401,29 @@ context where binding is lexical by default.  */)
 }
 
 \f
-DEFUN ("backtrace-debug", Fbacktrace_debug, Sbacktrace_debug, 2, 2, 0,
-       doc: /* Set the debug-on-exit flag of eval frame LEVEL levels down to FLAG.
-The debugger is entered when that frame exits, if the flag is non-nil.  */)
-  (Lisp_Object level, Lisp_Object flag)
-{
-  union specbinding *pdl = backtrace_top ();
-  register EMACS_INT i;
-
-  CHECK_NUMBER (level);
-
-  for (i = 0; backtrace_p (pdl) && i < XINT (level); i++)
-    pdl = backtrace_next (pdl);
-
-  if (backtrace_p (pdl))
-    set_backtrace_debug_on_exit (pdl, !NILP (flag));
-
-  return flag;
-}
-
-DEFUN ("backtrace", Fbacktrace, Sbacktrace, 0, 0, "",
-       doc: /* Print a trace of Lisp function calls currently active.
-Output stream used is value of `standard-output'.  */)
-  (void)
+static union specbinding *
+get_backtrace_starting_at (Lisp_Object base)
 {
   union specbinding *pdl = backtrace_top ();
-  Lisp_Object tem;
-  Lisp_Object old_print_level = Vprint_level;
-
-  if (NILP (Vprint_level))
-    XSETFASTINT (Vprint_level, 8);
 
-  while (backtrace_p (pdl))
-    {
-      write_string (backtrace_debug_on_exit (pdl) ? "* " : "  ");
-      if (backtrace_nargs (pdl) == UNEVALLED)
-	{
-	  Fprin1 (Fcons (backtrace_function (pdl), *backtrace_args (pdl)),
-		  Qnil);
-	  write_string ("\n");
-	}
-      else
-	{
-	  tem = backtrace_function (pdl);
-	  if (debugger_stack_frame_as_list)
-	    write_string ("(");
-	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
-	  if (!debugger_stack_frame_as_list)
-	    write_string ("(");
-	  {
-	    ptrdiff_t i;
-	    for (i = 0; i < backtrace_nargs (pdl); i++)
-	      {
-		if (i || debugger_stack_frame_as_list)
-		  write_string(" ");
-		Fprin1 (backtrace_args (pdl)[i], Qnil);
-	      }
-	  }
-	  write_string (")\n");
-	}
-      pdl = backtrace_next (pdl);
+  if (!NILP (base))
+    { /* Skip up to `base'.  */
+      base = Findirect_function (base, Qt);
+      while (backtrace_p (pdl)
+             && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
+        pdl = backtrace_next (pdl);
     }
 
-  Vprint_level = old_print_level;
-  return Qnil;
+  return pdl;
 }
 
 static union specbinding *
 get_backtrace_frame (Lisp_Object nframes, Lisp_Object base)
 {
-  union specbinding *pdl = backtrace_top ();
   register EMACS_INT i;
 
   CHECK_NATNUM (nframes);
-
-  if (!NILP (base))
-    { /* Skip up to `base'.  */
-      base = Findirect_function (base, Qt);
-      while (backtrace_p (pdl)
-	     && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
-	pdl = backtrace_next (pdl);
-    }
+  union specbinding *pdl = get_backtrace_starting_at (base);
 
   /* Find the frame requested.  */
   for (i = XFASTINT (nframes); i > 0 && backtrace_p (pdl); i--)
@@ -3490,33 +3432,74 @@ get_backtrace_frame (Lisp_Object nframes, Lisp_Object base)
   return pdl;
 }
 
-DEFUN ("backtrace-frame", Fbacktrace_frame, Sbacktrace_frame, 1, 2, NULL,
-       doc: /* Return the function and arguments NFRAMES up from current execution point.
-If that frame has not evaluated the arguments yet (or is a special form),
-the value is (nil FUNCTION ARG-FORMS...).
-If that frame has evaluated its arguments and called its function already,
-the value is (t FUNCTION ARG-VALUES...).
-A &rest arg is represented as the tail of the list ARG-VALUES.
-FUNCTION is whatever was supplied as car of evaluated list,
-or a lambda expression for macro calls.
-If NFRAMES is more than the number of frames, the value is nil.
-If BASE is non-nil, it should be a function and NFRAMES counts from its
-nearest activation frame.  */)
-  (Lisp_Object nframes, Lisp_Object base)
+Lisp_Object
+backtrace_frame_apply (Lisp_Object function, union specbinding *pdl)
 {
-  union specbinding *pdl = get_backtrace_frame (nframes, base);
-
   if (!backtrace_p (pdl))
     return Qnil;
+
+  Lisp_Object flags = Qnil;
+  if (backtrace_debug_on_exit (pdl))
+    {
+      flags = Fcons (QCdebug_on_exit, Fcons (Qt, Qnil));
+    }
+
   if (backtrace_nargs (pdl) == UNEVALLED)
-    return Fcons (Qnil,
-		  Fcons (backtrace_function (pdl), *backtrace_args (pdl)));
+    {
+      return call4 (function, Qnil, backtrace_function (pdl), *backtrace_args (pdl), flags);
+    }
   else
     {
       Lisp_Object tem = Flist (backtrace_nargs (pdl), backtrace_args (pdl));
+      return call4 (function, Qt, backtrace_function (pdl), tem, flags);
+    }
+}
 
-      return Fcons (Qt, Fcons (backtrace_function (pdl), tem));
+DEFUN ("backtrace-debug", Fbacktrace_debug, Sbacktrace_debug, 2, 2, 0,
+       doc: /* Set the debug-on-exit flag of eval frame LEVEL levels down to FLAG.
+The debugger is entered when that frame exits, if the flag is non-nil.  */)
+  (Lisp_Object level, Lisp_Object flag)
+{
+  CHECK_NUMBER (level);
+  union specbinding *pdl = get_backtrace_frame(level, Qnil);
+
+  if (backtrace_p (pdl))
+    set_backtrace_debug_on_exit (pdl, !NILP (flag));
+
+  return flag;
+}
+
+DEFUN ("mapbacktrace", Fmapbacktrace, Smapbacktrace, 1, 2, 0,
+       doc: /* Call FUNCTION for each frame in backtrace.
+FUNCTION is called with 4 arguments EVALD FUNC ARGS FLAGS.  If a frame
+has not evaluated its arguments yet or is a special form, EVALD is nil
+and ARGS is a list of forms.  If a frame has evaluated its arguments
+and called its function already, EVALD is t and ARGS is a list of
+values.  FLAGS is a plist of properties of the current frame:
+currently, the only supported property is :debug-on-exit.
+If BASE is non-nil, it should be a function and iteration will start
+from its nearest activation frame.
+`mapbacktrace' always returns nil.  */)
+     (Lisp_Object function, Lisp_Object base)
+{
+  union specbinding *pdl = get_backtrace_starting_at (base);
+
+  while (backtrace_p (pdl))
+    {
+      backtrace_frame_apply (function, pdl);
+      pdl = backtrace_next (pdl);
     }
+
+  return Qnil;
+}
+
+DEFUN ("backtrace-frame--internal", Fbacktrace_frame_internal,
+       Sbacktrace_frame_internal, 3, 3, NULL,
+       doc: /* Call FUNCTION on stack frame NFRAMES away from BASE.
+Return the result of FUNCTION, or nil if no matching frame could be found. */)
+     (Lisp_Object function, Lisp_Object nframes, Lisp_Object base)
+{
+  return backtrace_frame_apply (function, get_backtrace_frame (nframes, base));
 }
 
 /* For backtrace-eval, we want to temporarily unwind the last few elements of
@@ -3973,8 +3956,9 @@ alist of active lexical bindings.  */);
   defsubr (&Srun_hook_wrapped);
   defsubr (&Sfetch_bytecode);
   defsubr (&Sbacktrace_debug);
-  defsubr (&Sbacktrace);
-  defsubr (&Sbacktrace_frame);
+  DEFSYM (QCdebug_on_exit, ":debug-on-exit");
+  defsubr (&Smapbacktrace);
+  defsubr (&Sbacktrace_frame_internal);
   defsubr (&Sbacktrace_eval);
   defsubr (&Sbacktrace__locals);
   defsubr (&Sspecial_variable_p);
-- 
2.7.4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Mon, 5 Dec 2016 11:31:03 -0500
> 
> Help with the following warning would be much appreciated:
> 
>   eval.c:3436:1: warning: no previous prototype for ‘backtrace_frame_apply’ [-Wmissing-prototypes]
>    backtrace_frame_apply (Lisp_Object function, union specbinding *pdl)
>    ^
> 
> (why does this specific function cause this warning, while other newly introduced functions don't?)

The others are static, this one isn't.  If it really needs to be
callable from other source files, you need to put its prototype in
some header file, probably lisp.h.  Otherwise, make it static.

I will review the patch soon.  Thanks.

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Mon, 5 Dec 2016 09:14:38 -0500
> 
> On 2016-12-05 08:20, Stefan Monnier wrote:
> >> (defun backtrace ()
> >>   "Print a trace of Lisp function calls currently active.
> >> Output stream used is value of `standard-output'."
> >>   (mapbacktrace #'~/backtrace-1 1))
> > 
> > Have you tried it both byte-compiled and interpreted?  Maybe this
> > function is just simple enough that the result is the same in both
> > cases, but in my experience, the stack is sufficiently different in the
> > two cases that a constant nskip doesn't cut it (hence the use of `base`
> > in backtrace-frame).
> 
> Thanks; I attached an updated patch.  Removing `backtrace' from eval.c makes the patch much harder to read, so I'll do that later.

Thanks, allow me a few additional comments:

> ;; -*- lexical-binding: t -*-
> 
> (defun backtrace-1 (evald func args flags)
>   "Print a trace of a single stack frame to `standard-output'.
> EVALD, FUNC, ARGS, FLAGS are as in `mapbacktrace'."
>   (let ((print-level (or print-level 8)))
>     (princ (if (plist-get flags :debug-on-exit) "* " "  "))
>     (cond
>      ((and evald (not debugger-stack-frame-as-list))
>       (prin1 func)
>       (if args (prin1 args) (princ "()")))
>      (t
>       (prin1 (cons func args))))
>     (princ "\n")))
> 
> (defun backtrace ()
>   "Print a trace of Lisp function calls currently active.
> Output stream used is value of `standard-output'."
>   (mapbacktrace #'backtrace-1 'backtrace))
> 
> (backtrace)
> 
> (defun backtrace-frames ()
>   "Collect all frames of current backtrace into a list."
>   (let ((frames nil))
>     (mapbacktrace (lambda (&rest frame) (push frame frames)) 'backtrace-frames)
>     (nreverse frames)))
> 
> (backtrace-frames)
> 
> (defun ~/backtrace-frame (nframes &optional base)
>   "Return the function and arguments NFRAMES up from current execution point.
> If that frame has not evaluated the arguments yet (or is a special form),
> the value is (nil FUNCTION ARG-FORMS...).
> If that frame has evaluated its arguments and called its function already,
> the value is (t FUNCTION ARG-VALUES...).
> A &rest arg is represented as the tail of the list ARG-VALUES.
> FUNCTION is whatever was supplied as car of evaluated list,
> or a lambda expression for macro calls.
> If NFRAMES is more than the number of frames, the value is nil.
> If BASE is non-nil, it should be a function and NFRAMES counts from its
> nearest activation frame."

It is better to move the description of BASE to the 2nd line, as it's
an argument of this function, while the rest describes the details of
what the function does.  It is plausible that someone would like to
read the doc string just as a reminder of the API, so we had better
not force them to read the entire doc string.

>   (let ((frame nil))
>     (mapbacktrace (lambda (evald func args _)
>                     (when (and base (eq func base))
>                       (setq base nil))
>                     (unless base
>                       (when (eq nframes 0)
>                         (setq frame `(,evald ,func ,@args)))
>                       (setq nframes (1- nframes))))
>                   '~/backtrace-frame)
>     frame))

These functions should go to subr.el, I think.

> +DEFUN ("mapbacktrace", Fmapbacktrace, Smapbacktrace, 1, 2, 0,
> +       doc: /* Call FUNCTION for each frame in backtrace.

Likewise here: BASE is better described on the 2nd line.

> +FUNCTION is called with 4 arguments EVALD FUNC ARGS FLAGS.  If a frame
                                      ^
Colon ':' here, please.

> +      if (backtrace_debug_on_exit (pdl))
> +        {
> +          flags = Fcons (QCdebug_on_exit, Fcons (Qt, Qnil));
> +        }

No need for braces if there's only one statement in the 'if' clause.

> +      if (backtrace_nargs (pdl) == UNEVALLED)
> +        {
> +          call4 (function, Qnil, backtrace_function (pdl), *backtrace_args (pdl), flags);
> +        }

Same here.

Last, but not least: it would be nice to have a couple of tests for
this functionality.

Thanks again for working on this.

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 254 bytes --]

On 2016-12-05 11:23, Eli Zaretskii wrote:
> Thanks, allow me a few additional comments:
> [...]
> Last, but not least: it would be nice to have a couple of tests for
> this functionality.

Done and done :) See attached patch.

Cheers,
Clément.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-Move-backtrace-to-ELisp-using-a-new-mapbacktrace-pri.patch --]
[-- Type: text/x-diff; name="0001-Move-backtrace-to-ELisp-using-a-new-mapbacktrace-pri.patch", Size: 16673 bytes --]

From 3545cb4325e030a9f8480b4451d6a095aedbc479 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Mon, 5 Dec 2016 00:52:14 -0500
Subject: [PATCH] Move backtrace to ELisp using a new mapbacktrace primitive

* src/eval.c (get_backtrace_starting_at, backtrace_frame_apply)
(Fmapbacktrace, Fbacktrace_frame_internal): New functions.
(get_backtrace_frame, Fbacktrace_debug): Use `get_backtrace_starting_at'.

* lisp/subr.el (backtrace--print-frame): New function.
(backtrace): Reimplement using `backtrace--print-frame' and `mapbacktrace'.
(backtrace-frame): Reimplement using `backtrace-frame--internal'.

* lisp/emacs-lisp/debug.el (debugger-setup-buffer): Pass a base to
`mapbacktrace' instead of searching for "(debug" in the output of
`backtrace'.

* test/lisp/subr-tests.el (subr-test-backtrace-simple-tests)
(subr-test-backtrace-integration-test): New tests.

* doc/lispref/debugging.texi (Internals of Debugger): Document
`mapbacktrace' and missing argument BASE of `backtrace-frame'.
---
 doc/lispref/debugging.texi |  24 ++++++-
 etc/NEWS                   |   4 ++
 lisp/emacs-lisp/debug.el   |  11 ++--
 lisp/subr.el               |  45 +++++++++++++
 src/eval.c                 | 157 ++++++++++++++++++++-------------------------
 test/lisp/subr-tests.el    |  47 ++++++++++++++
 6 files changed, 193 insertions(+), 95 deletions(-)

diff --git a/doc/lispref/debugging.texi b/doc/lispref/debugging.texi
index c80b0f9..c05b0eb 100644
--- a/doc/lispref/debugging.texi
+++ b/doc/lispref/debugging.texi
@@ -727,7 +727,7 @@ Internals of Debugger
 This variable is obsolete and will be removed in future versions.
 @end defvar
 
-@defun backtrace-frame frame-number
+@defun backtrace-frame frame-number &optional base
 The function @code{backtrace-frame} is intended for use in Lisp
 debuggers.  It returns information about what computation is happening
 in the stack frame @var{frame-number} levels down.
@@ -744,10 +744,32 @@ Internals of Debugger
 case of a macro call.  If the function has a @code{&rest} argument, that
 is represented as the tail of the list @var{arg-values}.
 
+If @var{base} is specified, @var{frame-number} counts relative to
+the topmost frame whose @var{function} is @var{base}.
+
 If @var{frame-number} is out of range, @code{backtrace-frame} returns
 @code{nil}.
 @end defun
 
+@defun mapbacktrace function &optional base
+The function @code{mapbacktrace} calls @var{function} once for each
+frame in the backtrace, starting at the first frame whose
+@var{function} is base (or from the top if @var{base} is omitted or
+@code{nil}).
+
+@var{function} is called with four arguments: @var{evald}, @var{func},
+@var{args}, and @var{flags}.
+
+If a frame has not evaluated its arguments yet or is a special form,
+@var{evald} is @code{nil} and @var{args} is a list of forms.
+
+If a frame has evaluated its arguments and called its function
+already, @var{evald} is @code{t} and @var{args} is a list of values.
+@var{flags} is a plist of properties of the current frame: currently,
+the only supported property is @code{:debug-on-exit}, which is t if
+the frame's debug-on-exit flag is set.
+@end defun
+
 @include edebug.texi
 
 @node Syntax Errors
diff --git a/etc/NEWS b/etc/NEWS
index a62668a..72bef06 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -74,6 +74,10 @@ for '--daemon'.
 * Changes in Emacs 26.1
 
 +++
+** The new function 'mapbacktrace' applies a function to all frames of
+the current stack trace.
+
++++
 ** The new function 'file-name-case-insensitive-p' tests whether a
 given file is on a case-insensitive filesystem.
 
diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 5430b72..5a4b097 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -274,15 +274,14 @@ debugger-setup-buffer
   (let ((standard-output (current-buffer))
 	(print-escape-newlines t)
 	(print-level 8)
-	(print-length 50))
-    (backtrace))
+        (print-length 50))
+    ;; FIXME the debugger could pass a custom callback to mapbacktrace
+    ;; instead of manipulating printed results.
+    (mapbacktrace #'backtrace--print-frame 'debug))
   (goto-char (point-min))
   (delete-region (point)
 		 (progn
-		   (search-forward (if debugger-stack-frame-as-list
-                                       "\n  (debug "
-                                     "\n  debug("))
-		   (forward-line (if (eq (car args) 'debug)
+                   (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
 				     3
diff --git a/lisp/subr.el b/lisp/subr.el
index 5da5bf8..658472d 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -4333,6 +4333,51 @@ define-mail-user-agent
   (put symbol 'sendfunc sendfunc)
   (put symbol 'abortfunc (or abortfunc 'kill-buffer))
   (put symbol 'hookvar (or hookvar 'mail-send-hook)))
+
+\f
+(defun backtrace--print-frame (evald func args flags)
+  "Print a trace of a single stack frame to `standard-output'.
+EVALD, FUNC, ARGS, FLAGS are as in `mapbacktrace'."
+  (princ (if (plist-get flags :debug-on-exit) "* " "  "))
+  (cond
+   ((and evald (not debugger-stack-frame-as-list))
+    (prin1 func)
+    (if args (prin1 args) (princ "()")))
+   (t
+    (prin1 (cons func args))))
+  (princ "\n"))
+
+(defun backtrace ()
+  "Print a trace of Lisp function calls currently active.
+Output stream used is value of `standard-output'."
+  (let ((print-level (or print-level 8)))
+    (mapbacktrace #'backtrace--print-frame 'backtrace)))
+
+(defun backtrace-frames (&optional base)
+  "Collect all frames of current backtrace into a list.
+If non-nil BASE should be a function, and frames before its
+nearest activation frames are discarded."
+  (let ((frames nil))
+    (mapbacktrace (lambda (&rest frame) (push frame frames))
+                  (or base 'backtrace-frames))
+    (nreverse frames)))
+
+(defun backtrace-frame (nframes &optional base)
+  "Return the function and arguments NFRAMES up from current execution point.
+If non-nil BASE should be a function, and NFRAMES counts from its
+nearest activation frame.
+If the frame has not evaluated the arguments yet (or is a special form),
+the value is (nil FUNCTION ARG-FORMS...).
+If the frame has evaluated its arguments and called its function already,
+the value is (t FUNCTION ARG-VALUES...).
+A &rest arg is represented as the tail of the list ARG-VALUES.
+FUNCTION is whatever was supplied as car of evaluated list,
+or a lambda expression for macro calls.
+If NFRAMES is more than the number of frames, the value is nil."
+  (backtrace-frame--internal
+   (lambda (evald func args _) `(,evald ,func ,@args))
+   nframes (or base 'backtrace-frame)))
+
 \f
 (defvar called-interactively-p-functions nil
   "Special hook called to skip special frames in `called-interactively-p'.
diff --git a/src/eval.c b/src/eval.c
index 724f001..50c02e5 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3401,87 +3401,29 @@ context where binding is lexical by default.  */)
 }
 
 \f
-DEFUN ("backtrace-debug", Fbacktrace_debug, Sbacktrace_debug, 2, 2, 0,
-       doc: /* Set the debug-on-exit flag of eval frame LEVEL levels down to FLAG.
-The debugger is entered when that frame exits, if the flag is non-nil.  */)
-  (Lisp_Object level, Lisp_Object flag)
-{
-  union specbinding *pdl = backtrace_top ();
-  register EMACS_INT i;
-
-  CHECK_NUMBER (level);
-
-  for (i = 0; backtrace_p (pdl) && i < XINT (level); i++)
-    pdl = backtrace_next (pdl);
-
-  if (backtrace_p (pdl))
-    set_backtrace_debug_on_exit (pdl, !NILP (flag));
-
-  return flag;
-}
-
-DEFUN ("backtrace", Fbacktrace, Sbacktrace, 0, 0, "",
-       doc: /* Print a trace of Lisp function calls currently active.
-Output stream used is value of `standard-output'.  */)
-  (void)
+static union specbinding *
+get_backtrace_starting_at (Lisp_Object base)
 {
   union specbinding *pdl = backtrace_top ();
-  Lisp_Object tem;
-  Lisp_Object old_print_level = Vprint_level;
 
-  if (NILP (Vprint_level))
-    XSETFASTINT (Vprint_level, 8);
-
-  while (backtrace_p (pdl))
-    {
-      write_string (backtrace_debug_on_exit (pdl) ? "* " : "  ");
-      if (backtrace_nargs (pdl) == UNEVALLED)
-	{
-	  Fprin1 (Fcons (backtrace_function (pdl), *backtrace_args (pdl)),
-		  Qnil);
-	  write_string ("\n");
-	}
-      else
-	{
-	  tem = backtrace_function (pdl);
-	  if (debugger_stack_frame_as_list)
-	    write_string ("(");
-	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
-	  if (!debugger_stack_frame_as_list)
-	    write_string ("(");
-	  {
-	    ptrdiff_t i;
-	    for (i = 0; i < backtrace_nargs (pdl); i++)
-	      {
-		if (i || debugger_stack_frame_as_list)
-		  write_string(" ");
-		Fprin1 (backtrace_args (pdl)[i], Qnil);
-	      }
-	  }
-	  write_string (")\n");
-	}
-      pdl = backtrace_next (pdl);
+  if (!NILP (base))
+    { /* Skip up to `base'.  */
+      base = Findirect_function (base, Qt);
+      while (backtrace_p (pdl)
+             && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
+        pdl = backtrace_next (pdl);
     }
 
-  Vprint_level = old_print_level;
-  return Qnil;
+  return pdl;
 }
 
 static union specbinding *
 get_backtrace_frame (Lisp_Object nframes, Lisp_Object base)
 {
-  union specbinding *pdl = backtrace_top ();
   register EMACS_INT i;
 
   CHECK_NATNUM (nframes);
-
-  if (!NILP (base))
-    { /* Skip up to `base'.  */
-      base = Findirect_function (base, Qt);
-      while (backtrace_p (pdl)
-	     && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
-	pdl = backtrace_next (pdl);
-    }
+  union specbinding *pdl = get_backtrace_starting_at (base);
 
   /* Find the frame requested.  */
   for (i = XFASTINT (nframes); i > 0 && backtrace_p (pdl); i--)
@@ -3490,33 +3432,71 @@ get_backtrace_frame (Lisp_Object nframes, Lisp_Object base)
   return pdl;
 }
 
-DEFUN ("backtrace-frame", Fbacktrace_frame, Sbacktrace_frame, 1, 2, NULL,
-       doc: /* Return the function and arguments NFRAMES up from current execution point.
-If that frame has not evaluated the arguments yet (or is a special form),
-the value is (nil FUNCTION ARG-FORMS...).
-If that frame has evaluated its arguments and called its function already,
-the value is (t FUNCTION ARG-VALUES...).
-A &rest arg is represented as the tail of the list ARG-VALUES.
-FUNCTION is whatever was supplied as car of evaluated list,
-or a lambda expression for macro calls.
-If NFRAMES is more than the number of frames, the value is nil.
-If BASE is non-nil, it should be a function and NFRAMES counts from its
-nearest activation frame.  */)
-  (Lisp_Object nframes, Lisp_Object base)
+static Lisp_Object
+backtrace_frame_apply (Lisp_Object function, union specbinding *pdl)
 {
-  union specbinding *pdl = get_backtrace_frame (nframes, base);
-
   if (!backtrace_p (pdl))
     return Qnil;
+
+  Lisp_Object flags = Qnil;
+  if (backtrace_debug_on_exit (pdl))
+      flags = Fcons (QCdebug_on_exit, Fcons (Qt, Qnil));
+
   if (backtrace_nargs (pdl) == UNEVALLED)
-    return Fcons (Qnil,
-		  Fcons (backtrace_function (pdl), *backtrace_args (pdl)));
+      return call4 (function, Qnil, backtrace_function (pdl), *backtrace_args (pdl), flags);
   else
     {
       Lisp_Object tem = Flist (backtrace_nargs (pdl), backtrace_args (pdl));
+      return call4 (function, Qt, backtrace_function (pdl), tem, flags);
+    }
+}
 
-      return Fcons (Qt, Fcons (backtrace_function (pdl), tem));
+DEFUN ("backtrace-debug", Fbacktrace_debug, Sbacktrace_debug, 2, 2, 0,
+       doc: /* Set the debug-on-exit flag of eval frame LEVEL levels down to FLAG.
+The debugger is entered when that frame exits, if the flag is non-nil.  */)
+  (Lisp_Object level, Lisp_Object flag)
+{
+  CHECK_NUMBER (level);
+  union specbinding *pdl = get_backtrace_frame(level, Qnil);
+
+  if (backtrace_p (pdl))
+    set_backtrace_debug_on_exit (pdl, !NILP (flag));
+
+  return flag;
+}
+
+DEFUN ("mapbacktrace", Fmapbacktrace, Smapbacktrace, 1, 2, 0,
+       doc: /* Call FUNCTION for each frame in backtrace.
+If BASE is non-nil, it should be a function and iteration will start
+from its nearest activation frame.
+FUNCTION is called with 4 arguments: EVALD, FUNC, ARGS, and FLAGS.  If
+a frame has not evaluated its arguments yet or is a special form,
+EVALD is nil and ARGS is a list of forms.  If a frame has evaluated
+its arguments and called its function already, EVALD is t and ARGS is
+a list of values.
+FLAGS is a plist of properties of the current frame: currently, the
+only supported property is :debug-on-exit.  `mapbacktrace' always
+returns nil.  */)
+     (Lisp_Object function, Lisp_Object base)
+{
+  union specbinding *pdl = get_backtrace_starting_at (base);
+
+  while (backtrace_p (pdl))
+    {
+      backtrace_frame_apply (function, pdl);
+      pdl = backtrace_next (pdl);
     }
+
+  return Qnil;
+}
+
+DEFUN ("backtrace-frame--internal", Fbacktrace_frame_internal,
+       Sbacktrace_frame_internal, 3, 3, NULL,
+       doc: /* Call FUNCTION on stack frame NFRAMES away from BASE.
+Return the result of FUNCTION, or nil if no matching frame could be found. */)
+     (Lisp_Object function, Lisp_Object nframes, Lisp_Object base)
+{
+  return backtrace_frame_apply (function, get_backtrace_frame (nframes, base));
 }
 
 /* For backtrace-eval, we want to temporarily unwind the last few elements of
@@ -3973,8 +3953,9 @@ alist of active lexical bindings.  */);
   defsubr (&Srun_hook_wrapped);
   defsubr (&Sfetch_bytecode);
   defsubr (&Sbacktrace_debug);
-  defsubr (&Sbacktrace);
-  defsubr (&Sbacktrace_frame);
+  DEFSYM (QCdebug_on_exit, ":debug-on-exit");
+  defsubr (&Smapbacktrace);
+  defsubr (&Sbacktrace_frame_internal);
   defsubr (&Sbacktrace_eval);
   defsubr (&Sbacktrace__locals);
   defsubr (&Sspecial_variable_p);
diff --git a/test/lisp/subr-tests.el b/test/lisp/subr-tests.el
index ce21290..82a70ca 100644
--- a/test/lisp/subr-tests.el
+++ b/test/lisp/subr-tests.el
@@ -224,5 +224,52 @@
               (error-message-string (should-error (version-to-list "beta22_8alpha3")))
               "Invalid version syntax: `beta22_8alpha3' (must start with a number)"))))
 
+(defun subr-test--backtrace-frames-with-backtrace-frame (base)
+  "Reference implementation of `backtrace-frames'."
+  (let ((idx 0)
+        (frame nil)
+        (frames nil))
+    (while (setq frame (backtrace-frame idx base))
+      (push frame frames)
+      (setq idx (1+ idx)))
+    (nreverse frames)))
+
+(defun subr-test--frames-2 (base)
+  (let ((_dummy nil))
+    (progn ;; Add a few frames to top of stack
+      (unwind-protect
+          (cons (mapcar (pcase-lambda (`(,evald ,func ,args ,_))
+                          `(,evald ,func ,@args))
+                        (backtrace-frames base))
+                (subr-test--backtrace-frames-with-backtrace-frame base))))))
+
+(defun subr-test--frames-1 (base)
+  (subr-test--frames-2 base))
+
+(ert-deftest subr-test-backtrace-simple-tests ()
+  "Test backtrace-related functions (simple tests).
+This exercises `backtrace-frame', and indirectly `mapbacktrace'."
+  ;; `mapbacktrace' returns nil
+  (should (equal (mapbacktrace #'ignore) nil))
+  ;; Unbound BASE is silently ignored
+  (let ((unbound (make-symbol "ub")))
+    (should (equal (backtrace-frame 0 unbound) nil))
+    (should (equal (mapbacktrace #'error unbound) nil)))
+  ;; First frame is backtrace-related function
+  (should (equal (backtrace-frame 0) '(t backtrace-frame 0)))
+  (should (equal (catch 'ret
+                   (mapbacktrace (lambda (&rest args) (throw 'ret args))))
+                 '(t mapbacktrace ((lambda (&rest args) (throw 'ret args))) nil)))
+  ;; Past-end NFRAMES is silently ignored
+  (should (equal (backtrace-frame most-positive-fixnum) nil)))
+
+(ert-deftest subr-test-backtrace-integration-test ()
+  "Test backtrace-related functions (integration test).
+This exercises `backtrace-frame', `backtrace-frames', and
+indirectly `mapbacktrace'."
+  ;; Compare two implementations of backtrace-frames
+  (let ((frame-lists (subr-test--frames-1 'subr-test--frames-2)))
+    (should (equal (car frame-lists) (cdr frame-lists)))))
+
 (provide 'subr-tests)
 ;;; subr-tests.el ends here
-- 
2.7.4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Mon, 5 Dec 2016 13:59:03 -0500
> 
> Done and done :) See attached patch.

Thanks.

> +@defun mapbacktrace function &optional base
> +The function @code{mapbacktrace} calls @var{function} once for each
> +frame in the backtrace, starting at the first frame whose
> +@var{function} is base (or from the top if @var{base} is omitted or
   ^^^^^^^^^^^^^^^^^^^^^^
Here, "function" should not have the @var markup, and "base" should.

> +the only supported property is @code{:debug-on-exit}, which is t if
                                                                  ^
@code{t}

> +the frame's debug-on-exit flag is set.
               ^^^^^^^^^^^^^
@code{debug-on-exit}

Also, suggest to say "stack frame's", to avoid confusion with the
frames on display.

> +(defun backtrace-frames (&optional base)
> +  "Collect all frames of current backtrace into a list.
> +If non-nil BASE should be a function, and frames before its
             ^
Comma here.

> +(defun backtrace-frame (nframes &optional base)
> +  "Return the function and arguments NFRAMES up from current execution point.
> +If non-nil BASE should be a function, and NFRAMES counts from its
             ^
Likewise.

> +  if (backtrace_debug_on_exit (pdl))
> +      flags = Fcons (QCdebug_on_exit, Fcons (Qt, Qnil));
     ^^^^
Too many blanks here.  Should be only 2.

Otherwise, LGTM.

Thanks for doing this!

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 202 bytes --]

On 2016-12-06 13:55, Eli Zaretskii wrote:
> […]
> Otherwise, LGTM.

Thanks a lot for the review! I've attached an updated patch, which I'll to master in a few days if no one objects :)

Clément.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-Move-backtrace-to-ELisp-using-a-new-mapbacktrace-pri.patch --]
[-- Type: text/x-diff; name="0001-Move-backtrace-to-ELisp-using-a-new-mapbacktrace-pri.patch", Size: 16689 bytes --]

From 0a525bc06b992c2995dd8f5853f9485588a2bf88 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Mon, 5 Dec 2016 00:52:14 -0500
Subject: [PATCH] Move backtrace to ELisp using a new mapbacktrace primitive

* src/eval.c (get_backtrace_starting_at, backtrace_frame_apply)
(Fmapbacktrace, Fbacktrace_frame_internal): New functions.
(get_backtrace_frame, Fbacktrace_debug): Use `get_backtrace_starting_at'.

* lisp/subr.el (backtrace--print-frame): New function.
(backtrace): Reimplement using `backtrace--print-frame' and `mapbacktrace'.
(backtrace-frame): Reimplement using `backtrace-frame--internal'.

* lisp/emacs-lisp/debug.el (debugger-setup-buffer): Pass a base to
`mapbacktrace' instead of searching for "(debug" in the output of
`backtrace'.

* test/lisp/subr-tests.el (subr-test-backtrace-simple-tests)
(subr-test-backtrace-integration-test): New tests.

* doc/lispref/debugging.texi (Internals of Debugger): Document
`mapbacktrace' and missing argument BASE of `backtrace-frame'.
---
 doc/lispref/debugging.texi |  23 ++++++-
 etc/NEWS                   |   4 ++
 lisp/emacs-lisp/debug.el   |  11 ++--
 lisp/subr.el               |  45 +++++++++++++
 src/eval.c                 | 157 ++++++++++++++++++++-------------------------
 test/lisp/subr-tests.el    |  47 ++++++++++++++
 6 files changed, 192 insertions(+), 95 deletions(-)

diff --git a/doc/lispref/debugging.texi b/doc/lispref/debugging.texi
index c80b0f9..8fb663d 100644
--- a/doc/lispref/debugging.texi
+++ b/doc/lispref/debugging.texi
@@ -727,7 +727,7 @@ Internals of Debugger
 This variable is obsolete and will be removed in future versions.
 @end defvar
 
-@defun backtrace-frame frame-number
+@defun backtrace-frame frame-number &optional base
 The function @code{backtrace-frame} is intended for use in Lisp
 debuggers.  It returns information about what computation is happening
 in the stack frame @var{frame-number} levels down.
@@ -744,10 +744,31 @@ Internals of Debugger
 case of a macro call.  If the function has a @code{&rest} argument, that
 is represented as the tail of the list @var{arg-values}.
 
+If @var{base} is specified, @var{frame-number} counts relative to
+the topmost frame whose @var{function} is @var{base}.
+
 If @var{frame-number} is out of range, @code{backtrace-frame} returns
 @code{nil}.
 @end defun
 
+@defun mapbacktrace function &optional base
+The function @code{mapbacktrace} calls @var{function} once for each
+frame in the backtrace, starting at the first frame whose function is
+@var{base} (or from the top if @var{base} is omitted or @code{nil}).
+
+@var{function} is called with four arguments: @var{evald}, @var{func},
+@var{args}, and @var{flags}.
+
+If a frame has not evaluated its arguments yet or is a special form,
+@var{evald} is @code{nil} and @var{args} is a list of forms.
+
+If a frame has evaluated its arguments and called its function
+already, @var{evald} is @code{t} and @var{args} is a list of values.
+@var{flags} is a plist of properties of the current frame: currently,
+the only supported property is @code{:debug-on-exit}, which is
+@code{t} if the stack frame's @code{debug-on-exit} flag is set.
+@end defun
+
 @include edebug.texi
 
 @node Syntax Errors
diff --git a/etc/NEWS b/etc/NEWS
index a62668a..72bef06 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -74,6 +74,10 @@ for '--daemon'.
 * Changes in Emacs 26.1
 
 +++
+** The new function 'mapbacktrace' applies a function to all frames of
+the current stack trace.
+
++++
 ** The new function 'file-name-case-insensitive-p' tests whether a
 given file is on a case-insensitive filesystem.
 
diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 5430b72..5a4b097 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -274,15 +274,14 @@ debugger-setup-buffer
   (let ((standard-output (current-buffer))
 	(print-escape-newlines t)
 	(print-level 8)
-	(print-length 50))
-    (backtrace))
+        (print-length 50))
+    ;; FIXME the debugger could pass a custom callback to mapbacktrace
+    ;; instead of manipulating printed results.
+    (mapbacktrace #'backtrace--print-frame 'debug))
   (goto-char (point-min))
   (delete-region (point)
 		 (progn
-		   (search-forward (if debugger-stack-frame-as-list
-                                       "\n  (debug "
-                                     "\n  debug("))
-		   (forward-line (if (eq (car args) 'debug)
+                   (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
 				     3
diff --git a/lisp/subr.el b/lisp/subr.el
index 5da5bf8..6ab1d5f 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -4333,6 +4333,51 @@ define-mail-user-agent
   (put symbol 'sendfunc sendfunc)
   (put symbol 'abortfunc (or abortfunc 'kill-buffer))
   (put symbol 'hookvar (or hookvar 'mail-send-hook)))
+
+\f
+(defun backtrace--print-frame (evald func args flags)
+  "Print a trace of a single stack frame to `standard-output'.
+EVALD, FUNC, ARGS, FLAGS are as in `mapbacktrace'."
+  (princ (if (plist-get flags :debug-on-exit) "* " "  "))
+  (cond
+   ((and evald (not debugger-stack-frame-as-list))
+    (prin1 func)
+    (if args (prin1 args) (princ "()")))
+   (t
+    (prin1 (cons func args))))
+  (princ "\n"))
+
+(defun backtrace ()
+  "Print a trace of Lisp function calls currently active.
+Output stream used is value of `standard-output'."
+  (let ((print-level (or print-level 8)))
+    (mapbacktrace #'backtrace--print-frame 'backtrace)))
+
+(defun backtrace-frames (&optional base)
+  "Collect all frames of current backtrace into a list.
+If non-nil, BASE should be a function, and frames before its
+nearest activation frames are discarded."
+  (let ((frames nil))
+    (mapbacktrace (lambda (&rest frame) (push frame frames))
+                  (or base 'backtrace-frames))
+    (nreverse frames)))
+
+(defun backtrace-frame (nframes &optional base)
+  "Return the function and arguments NFRAMES up from current execution point.
+If non-nil, BASE should be a function, and NFRAMES counts from its
+nearest activation frame.
+If the frame has not evaluated the arguments yet (or is a special form),
+the value is (nil FUNCTION ARG-FORMS...).
+If the frame has evaluated its arguments and called its function already,
+the value is (t FUNCTION ARG-VALUES...).
+A &rest arg is represented as the tail of the list ARG-VALUES.
+FUNCTION is whatever was supplied as car of evaluated list,
+or a lambda expression for macro calls.
+If NFRAMES is more than the number of frames, the value is nil."
+  (backtrace-frame--internal
+   (lambda (evald func args _) `(,evald ,func ,@args))
+   nframes (or base 'backtrace-frame)))
+
 \f
 (defvar called-interactively-p-functions nil
   "Special hook called to skip special frames in `called-interactively-p'.
diff --git a/src/eval.c b/src/eval.c
index 724f001..929b942 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3401,87 +3401,29 @@ context where binding is lexical by default.  */)
 }
 
 \f
-DEFUN ("backtrace-debug", Fbacktrace_debug, Sbacktrace_debug, 2, 2, 0,
-       doc: /* Set the debug-on-exit flag of eval frame LEVEL levels down to FLAG.
-The debugger is entered when that frame exits, if the flag is non-nil.  */)
-  (Lisp_Object level, Lisp_Object flag)
-{
-  union specbinding *pdl = backtrace_top ();
-  register EMACS_INT i;
-
-  CHECK_NUMBER (level);
-
-  for (i = 0; backtrace_p (pdl) && i < XINT (level); i++)
-    pdl = backtrace_next (pdl);
-
-  if (backtrace_p (pdl))
-    set_backtrace_debug_on_exit (pdl, !NILP (flag));
-
-  return flag;
-}
-
-DEFUN ("backtrace", Fbacktrace, Sbacktrace, 0, 0, "",
-       doc: /* Print a trace of Lisp function calls currently active.
-Output stream used is value of `standard-output'.  */)
-  (void)
+static union specbinding *
+get_backtrace_starting_at (Lisp_Object base)
 {
   union specbinding *pdl = backtrace_top ();
-  Lisp_Object tem;
-  Lisp_Object old_print_level = Vprint_level;
 
-  if (NILP (Vprint_level))
-    XSETFASTINT (Vprint_level, 8);
-
-  while (backtrace_p (pdl))
-    {
-      write_string (backtrace_debug_on_exit (pdl) ? "* " : "  ");
-      if (backtrace_nargs (pdl) == UNEVALLED)
-	{
-	  Fprin1 (Fcons (backtrace_function (pdl), *backtrace_args (pdl)),
-		  Qnil);
-	  write_string ("\n");
-	}
-      else
-	{
-	  tem = backtrace_function (pdl);
-	  if (debugger_stack_frame_as_list)
-	    write_string ("(");
-	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
-	  if (!debugger_stack_frame_as_list)
-	    write_string ("(");
-	  {
-	    ptrdiff_t i;
-	    for (i = 0; i < backtrace_nargs (pdl); i++)
-	      {
-		if (i || debugger_stack_frame_as_list)
-		  write_string(" ");
-		Fprin1 (backtrace_args (pdl)[i], Qnil);
-	      }
-	  }
-	  write_string (")\n");
-	}
-      pdl = backtrace_next (pdl);
+  if (!NILP (base))
+    { /* Skip up to `base'.  */
+      base = Findirect_function (base, Qt);
+      while (backtrace_p (pdl)
+             && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
+        pdl = backtrace_next (pdl);
     }
 
-  Vprint_level = old_print_level;
-  return Qnil;
+  return pdl;
 }
 
 static union specbinding *
 get_backtrace_frame (Lisp_Object nframes, Lisp_Object base)
 {
-  union specbinding *pdl = backtrace_top ();
   register EMACS_INT i;
 
   CHECK_NATNUM (nframes);
-
-  if (!NILP (base))
-    { /* Skip up to `base'.  */
-      base = Findirect_function (base, Qt);
-      while (backtrace_p (pdl)
-	     && !EQ (base, Findirect_function (backtrace_function (pdl), Qt)))
-	pdl = backtrace_next (pdl);
-    }
+  union specbinding *pdl = get_backtrace_starting_at (base);
 
   /* Find the frame requested.  */
   for (i = XFASTINT (nframes); i > 0 && backtrace_p (pdl); i--)
@@ -3490,33 +3432,71 @@ get_backtrace_frame (Lisp_Object nframes, Lisp_Object base)
   return pdl;
 }
 
-DEFUN ("backtrace-frame", Fbacktrace_frame, Sbacktrace_frame, 1, 2, NULL,
-       doc: /* Return the function and arguments NFRAMES up from current execution point.
-If that frame has not evaluated the arguments yet (or is a special form),
-the value is (nil FUNCTION ARG-FORMS...).
-If that frame has evaluated its arguments and called its function already,
-the value is (t FUNCTION ARG-VALUES...).
-A &rest arg is represented as the tail of the list ARG-VALUES.
-FUNCTION is whatever was supplied as car of evaluated list,
-or a lambda expression for macro calls.
-If NFRAMES is more than the number of frames, the value is nil.
-If BASE is non-nil, it should be a function and NFRAMES counts from its
-nearest activation frame.  */)
-  (Lisp_Object nframes, Lisp_Object base)
+static Lisp_Object
+backtrace_frame_apply (Lisp_Object function, union specbinding *pdl)
 {
-  union specbinding *pdl = get_backtrace_frame (nframes, base);
-
   if (!backtrace_p (pdl))
     return Qnil;
+
+  Lisp_Object flags = Qnil;
+  if (backtrace_debug_on_exit (pdl))
+    flags = Fcons (QCdebug_on_exit, Fcons (Qt, Qnil));
+
   if (backtrace_nargs (pdl) == UNEVALLED)
-    return Fcons (Qnil,
-		  Fcons (backtrace_function (pdl), *backtrace_args (pdl)));
+    return call4 (function, Qnil, backtrace_function (pdl), *backtrace_args (pdl), flags);
   else
     {
       Lisp_Object tem = Flist (backtrace_nargs (pdl), backtrace_args (pdl));
+      return call4 (function, Qt, backtrace_function (pdl), tem, flags);
+    }
+}
 
-      return Fcons (Qt, Fcons (backtrace_function (pdl), tem));
+DEFUN ("backtrace-debug", Fbacktrace_debug, Sbacktrace_debug, 2, 2, 0,
+       doc: /* Set the debug-on-exit flag of eval frame LEVEL levels down to FLAG.
+The debugger is entered when that frame exits, if the flag is non-nil.  */)
+  (Lisp_Object level, Lisp_Object flag)
+{
+  CHECK_NUMBER (level);
+  union specbinding *pdl = get_backtrace_frame(level, Qnil);
+
+  if (backtrace_p (pdl))
+    set_backtrace_debug_on_exit (pdl, !NILP (flag));
+
+  return flag;
+}
+
+DEFUN ("mapbacktrace", Fmapbacktrace, Smapbacktrace, 1, 2, 0,
+       doc: /* Call FUNCTION for each frame in backtrace.
+If BASE is non-nil, it should be a function and iteration will start
+from its nearest activation frame.
+FUNCTION is called with 4 arguments: EVALD, FUNC, ARGS, and FLAGS.  If
+a frame has not evaluated its arguments yet or is a special form,
+EVALD is nil and ARGS is a list of forms.  If a frame has evaluated
+its arguments and called its function already, EVALD is t and ARGS is
+a list of values.
+FLAGS is a plist of properties of the current frame: currently, the
+only supported property is :debug-on-exit.  `mapbacktrace' always
+returns nil.  */)
+     (Lisp_Object function, Lisp_Object base)
+{
+  union specbinding *pdl = get_backtrace_starting_at (base);
+
+  while (backtrace_p (pdl))
+    {
+      backtrace_frame_apply (function, pdl);
+      pdl = backtrace_next (pdl);
     }
+
+  return Qnil;
+}
+
+DEFUN ("backtrace-frame--internal", Fbacktrace_frame_internal,
+       Sbacktrace_frame_internal, 3, 3, NULL,
+       doc: /* Call FUNCTION on stack frame NFRAMES away from BASE.
+Return the result of FUNCTION, or nil if no matching frame could be found. */)
+     (Lisp_Object function, Lisp_Object nframes, Lisp_Object base)
+{
+  return backtrace_frame_apply (function, get_backtrace_frame (nframes, base));
 }
 
 /* For backtrace-eval, we want to temporarily unwind the last few elements of
@@ -3973,8 +3953,9 @@ alist of active lexical bindings.  */);
   defsubr (&Srun_hook_wrapped);
   defsubr (&Sfetch_bytecode);
   defsubr (&Sbacktrace_debug);
-  defsubr (&Sbacktrace);
-  defsubr (&Sbacktrace_frame);
+  DEFSYM (QCdebug_on_exit, ":debug-on-exit");
+  defsubr (&Smapbacktrace);
+  defsubr (&Sbacktrace_frame_internal);
   defsubr (&Sbacktrace_eval);
   defsubr (&Sbacktrace__locals);
   defsubr (&Sspecial_variable_p);
diff --git a/test/lisp/subr-tests.el b/test/lisp/subr-tests.el
index ce21290..82a70ca 100644
--- a/test/lisp/subr-tests.el
+++ b/test/lisp/subr-tests.el
@@ -224,5 +224,52 @@
               (error-message-string (should-error (version-to-list "beta22_8alpha3")))
               "Invalid version syntax: `beta22_8alpha3' (must start with a number)"))))
 
+(defun subr-test--backtrace-frames-with-backtrace-frame (base)
+  "Reference implementation of `backtrace-frames'."
+  (let ((idx 0)
+        (frame nil)
+        (frames nil))
+    (while (setq frame (backtrace-frame idx base))
+      (push frame frames)
+      (setq idx (1+ idx)))
+    (nreverse frames)))
+
+(defun subr-test--frames-2 (base)
+  (let ((_dummy nil))
+    (progn ;; Add a few frames to top of stack
+      (unwind-protect
+          (cons (mapcar (pcase-lambda (`(,evald ,func ,args ,_))
+                          `(,evald ,func ,@args))
+                        (backtrace-frames base))
+                (subr-test--backtrace-frames-with-backtrace-frame base))))))
+
+(defun subr-test--frames-1 (base)
+  (subr-test--frames-2 base))
+
+(ert-deftest subr-test-backtrace-simple-tests ()
+  "Test backtrace-related functions (simple tests).
+This exercises `backtrace-frame', and indirectly `mapbacktrace'."
+  ;; `mapbacktrace' returns nil
+  (should (equal (mapbacktrace #'ignore) nil))
+  ;; Unbound BASE is silently ignored
+  (let ((unbound (make-symbol "ub")))
+    (should (equal (backtrace-frame 0 unbound) nil))
+    (should (equal (mapbacktrace #'error unbound) nil)))
+  ;; First frame is backtrace-related function
+  (should (equal (backtrace-frame 0) '(t backtrace-frame 0)))
+  (should (equal (catch 'ret
+                   (mapbacktrace (lambda (&rest args) (throw 'ret args))))
+                 '(t mapbacktrace ((lambda (&rest args) (throw 'ret args))) nil)))
+  ;; Past-end NFRAMES is silently ignored
+  (should (equal (backtrace-frame most-positive-fixnum) nil)))
+
+(ert-deftest subr-test-backtrace-integration-test ()
+  "Test backtrace-related functions (integration test).
+This exercises `backtrace-frame', `backtrace-frames', and
+indirectly `mapbacktrace'."
+  ;; Compare two implementations of backtrace-frames
+  (let ((frame-lists (subr-test--frames-1 'subr-test--frames-2)))
+    (should (equal (car frame-lists) (cdr frame-lists)))))
+
 (provide 'subr-tests)
 ;;; subr-tests.el ends here
-- 
2.7.4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Lisp-friendly backtraces [was: Lispy backtraces]

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 157 bytes --]

On 2016-12-07 03:27, Clément Pit--Claudel wrote:
> I've attached an updated patch, which I'll to push master in a few days if no one objects :)

Done!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 29 bytes --]

Ouch, forgot the attachment.

[-- Attachment #2: 0001-Make-backtraces-great-again.patch --]
[-- Type: text/x-diff, Size: 1444 bytes --]

From f85845921c5e06475c925508ba41b06340f4c95a Mon Sep 17 00:00:00 2001
From: Vasilij Schneidermann <v.schneidermann@gmail.com>
Date: Thu, 22 Sep 2016 23:01:21 +0200
Subject: [PATCH] Make backtraces great again

---
 lisp/emacs-lisp/debug.el | 2 +-
 src/eval.c               | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 22a3f39..a1ca336 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -279,7 +279,7 @@ That buffer should be current already."
   (goto-char (point-min))
   (delete-region (point)
 		 (progn
-		   (search-forward "\n  debug(")
+		   (search-forward "\n  (debug ")
 		   (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
diff --git a/src/eval.c b/src/eval.c
index 72facd5..698e0a1 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3409,13 +3409,13 @@ Output stream used is value of `standard-output'.  */)
       else
 	{
 	  tem = backtrace_function (pdl);
-	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
 	  write_string ("(");
+	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
 	  {
 	    ptrdiff_t i;
 	    for (i = 0; i < backtrace_nargs (pdl); i++)
 	      {
-		if (i) write_string (" ");
+		write_string (" ");
 		Fprin1 (backtrace_args (pdl)[i], Qnil);
 	      }
 	  }
-- 
2.9.3

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Thu, 22 Sep 2016 22:22:13 -0400
> 
> I'm not sure what the right way to transition is.  Maybe Emacs should let Lisp programs access the backtraces in a structured way, and then backtrace printing would only be a user-facing facility (programs wouldn't use the textual representation).

How about if we install this with a user option, off by default, that
switches to the new format when turned on?  Then we could collect user
experience for some time, and make this the default if everyone likes
it.

bug#24515: bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Vasilij Schneidermann]

> How about if we install this with a user option, off by default, that
> switches to the new format when turned on?

This sounds sensible, but I'm not sure whether it's worth complicating
the patch by adding checks for this new user option in `eval.c`,
`debug.el` and `edebug.el`.

> Then we could collect user experience for some time, and make this the
> default if everyone likes it.

How do you plan on informing and surveying users?  Would a discussion on
emacs-devel be the way to do this or do you have something different in
mind?

bug#24515: bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Eli Zaretskii]

> Date: Fri, 23 Sep 2016 11:55:41 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> Cc: Clément Pit--Claudel <clement.pit@gmail.com>
> 
> > How about if we install this with a user option, off by default, that
> > switches to the new format when turned on?
> 
> This sounds sensible, but I'm not sure whether it's worth complicating
> the patch by adding checks for this new user option in `eval.c`,
> `debug.el` and `edebug.el`.

It's a complication, sure.  But it doesn't sound too complicated, and
given the fears of breaking someone's code, I think it's justified.

> > Then we could collect user experience for some time, and make this the
> > default if everyone likes it.
> 
> How do you plan on informing and surveying users?  Would a discussion on
> emacs-devel be the way to do this or do you have something different in
> mind?

I thought just watching the discussions and the bug tracker would be
enough.  Then, a couple of releases from now, we could ask whether
making it the default would be okay, and see how people react.

bug#24515: bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 307 bytes --]

On 2016-09-23 05:55, Eli Zaretskii <eliz@gnu.org> wrote:
> How about if we install this with a user option, off by default, that
> switches to the new format when turned on?

I like this idea :) In that case, we should probably mention the options in NEWS, adding a note that code should be updated.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#24515: bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[John Wiegley]

>>>>> "CP" == Clément Pit--Claudel <clement.pit@gmail.com> writes:

PC> On 2016-09-23 05:55, Eli Zaretskii <eliz@gnu.org> wrote:
>> How about if we install this with a user option, off by default, that
>> switches to the new format when turned on?

CP> I like this idea :) In that case, we should probably mention the options
PC> in NEWS, adding a note that code should be updated.

I third the appreciation of having a user option.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Alan Mackenzie]

In article <mailman.2864.1474586229.22741.bug-gnu-emacs@gnu.org> you wrote:
> [-- text/plain, encoding 7bit, charset: utf-8, 30 lines --]

> I wrote a minimal patch that increases the overall consistency in a
> backtrace buffer by printing the call stack frames as S-Expressions.

> Before:

> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
>   +(1 t)
>   eval((+ 1 t) nil)
>   eval-expression((+ 1 t) nil)
>   call-interactively(eval-expression nil nil)
>   command-execute(eval-expression)

> After:

> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
>   (debug error (wrong-type-argument number-or-marker-p t))
>   (+ 1 t)
>   (eval (+ 1 t) nil)
>   (eval-expression (+ 1 t) nil)
>   (funcall-interactively eval-expression (+ 1 t) nil)
>   (call-interactively eval-expression nil nil)
>   (command-execute eval-expression)

I'm not sure I'm in favour of this change.  There is some tool in some
circumstances which prints the lines in the "before:" fashion
interspersed with internal forms from function which start off with "("
in column 0.  Having the distinction between lines starting with "(" and
lines starting with the function name is handy for telling them apart.

Sorry I can't be more specific about the circumstances this happens in,
but it happens relatively frequently.

> Now, this patch isn't perfect.  For some reason there's an extra debug
> line in the second version, I've yet to investigate into the reason for
> this.  The other problem is that while I can't imagine any reason to go
> back to the original view of the backtrace, I cannot rule out that this
> change might break other tools relying on it.  I'd appreciate any
> feedback on this.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

An item in a backtrace consists of a function and ACTUAL parameters
(argument values).  If you do  (list 'a 'b), the function will be  list
and the argument values will be (a b).

(list a b) is a misleading way to represent that.  It looks like a and b
are expressions to be evaluated.
-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 192 bytes --]

Hello again,

I've updated the patch as suggested to make the behavior configurable.
I'm pretty sure though that the customization option's name is terrible.
Any suggestions for a better one?

[-- Attachment #2: 0001-Make-backtraces-great-again.patch --]
[-- Type: text/x-diff, Size: 2739 bytes --]

From 7b953b3ec64b26edd812d0ad3e4e01bad4f94193 Mon Sep 17 00:00:00 2001
From: Vasilij Schneidermann <v.schneidermann@gmail.com>
Date: Thu, 22 Sep 2016 23:01:21 +0200
Subject: [PATCH] Make backtraces great again

---
 lisp/emacs-lisp/debug.el  |  4 +++-
 lisp/emacs-lisp/edebug.el |  4 +++-
 src/eval.c                | 12 ++++++++++--
 3 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 22a3f39..c481ee2 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -279,7 +279,9 @@ That buffer should be current already."
   (goto-char (point-min))
   (delete-region (point)
 		 (progn
-		   (search-forward "\n  debug(")
+		   (search-forward (if debugger-lispy-backtrace
+                                       "\n  (debug "
+                                     "\n  debug("))
 		   (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index c283c16..a2e9c00 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -3797,7 +3797,9 @@ Otherwise call `debug' normally."
 	  (forward-line 1)
 	  (delete-region last-ok-point (point)))
 
-	 ((looking-at "^  edebug")
+	 ((looking-at (if debugger-lispy-backtrace
+                          "^  (edebug"
+                        "^  edebug"))
 	  (forward-line 1)
 	  (delete-region last-ok-point (point))
 	  )))
diff --git a/src/eval.c b/src/eval.c
index 72facd5..fc6e5a6 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3409,13 +3409,17 @@ Output stream used is value of `standard-output'.  */)
       else
 	{
 	  tem = backtrace_function (pdl);
+	  if (debugger_lispy_backtrace)
+	    write_string ("(");
 	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
-	  write_string ("(");
+	  if (!debugger_lispy_backtrace)
+	    write_string ("(");
 	  {
 	    ptrdiff_t i;
 	    for (i = 0; i < backtrace_nargs (pdl); i++)
 	      {
-		if (i) write_string (" ");
+		if (i || debugger_lispy_backtrace)
+		  write_string(" ");
 		Fprin1 (backtrace_args (pdl)[i], Qnil);
 	      }
 	  }
@@ -3838,6 +3842,10 @@ This is nil when the debugger is called under circumstances where it
 might not be safe to continue.  */);
   debugger_may_continue = 1;
 
+  DEFVAR_BOOL ("debugger-lispy-backtrace", debugger_lispy_backtrace,
+	       doc: /* Non-nil means display call stack frames as lists. */);
+  debugger_lispy_backtrace = 0;
+
   DEFVAR_LISP ("debugger", Vdebugger,
 	       doc: /* Function to call to invoke debugger.
 If due to frame exit, args are `exit' and the value being returned;
-- 
2.9.3

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Eli Zaretskii]

> Date: Tue, 27 Sep 2016 21:16:03 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> 
> I've updated the patch as suggested to make the behavior configurable.

Thanks!

> I'm pretty sure though that the customization option's name is terrible.
> Any suggestions for a better one?

How about debugger-stack-frame-as-list?

Two minor nits to make this patch perfect:

  . Add the variable to cus-start.el, so that it will be a defcustom
  . Add a NEWS entry and mention the variable in the ELisp manual

Thanks a lot for working on this.

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 394 bytes --]

> How about debugger-stack-frame-as-list?

Thanks, sounds good to me.

> Two minor nits to make this patch perfect:
> 
>   . Add the variable to cus-start.el, so that it will be a defcustom
>   . Add a NEWS entry and mention the variable in the ELisp manual

I've done these and changed the commit message to comply better with the
changelog style.  Let me know if there's still something off.

[-- Attachment #2: 0001-Add-new-debugger-stack-frame-as-list-option.patch --]
[-- Type: text/x-diff, Size: 5421 bytes --]

From 62d47f75adc8d319b6fc35bb79bcf316d16bec00 Mon Sep 17 00:00:00 2001
From: Vasilij Schneidermann <v.schneidermann@gmail.com>
Date: Thu, 22 Sep 2016 23:01:21 +0200
Subject: [PATCH] Add new 'debugger-stack-frame-as-list' option

Allow displaying all lines in `backtrace' as lists.
* doc/lispref/debugging.texi: Document the new variable
* src/eval.c: New variable
* lisp/emacs-lisp/debug.el: Adjust backtrace processing for new variable
* lisp/emacs-lisp/edebug.el: Adjust backtrace processing for new
variable
---
 doc/lispref/debugging.texi | 29 +++++++++++++++++++++++++++++
 etc/NEWS                   |  5 +++++
 lisp/cus-start.el          |  1 +
 lisp/emacs-lisp/debug.el   |  4 +++-
 lisp/emacs-lisp/edebug.el  |  4 +++-
 src/eval.c                 | 12 ++++++++++--
 6 files changed, 51 insertions(+), 4 deletions(-)

diff --git a/doc/lispref/debugging.texi b/doc/lispref/debugging.texi
index 98c4705..15eea8a 100644
--- a/doc/lispref/debugging.texi
+++ b/doc/lispref/debugging.texi
@@ -623,6 +623,35 @@ forms are elided.
 @end smallexample
 @end deffn
 
+@defvar debugger-stack-frame-as-list
+If this variable is non-@code{nil}, every line of the backtrace is
+displayed as a list.  This aims to improve backtrace readability at
+the cost of special forms no longer being visually different from
+regular function calls.
+
+The above example would look as follows:
+@smallexample
+@group
+----------- Buffer: backtrace-output ------------
+  (backtrace)
+  (list ...computing arguments...)
+@end group
+  (progn ...)
+  (eval (progn (1+ var) (list (quote testing) (backtrace))))
+  (setq ...)
+  (save-excursion ...)
+  (let ...)
+  (with-output-to-temp-buffer ...)
+  (eval (with-output-to-temp-buffer ...))
+  (eval-last-sexp-1 nil)
+@group
+  (eval-last-sexp nil)
+  (call-interactively eval-last-sexp)
+----------- Buffer: backtrace-output ------------
+@end group
+@end smallexample
+@end defvar
+
 @defvar debug-on-next-call
 @cindex @code{eval}, and debugging
 @cindex @code{apply}, and debugging
diff --git a/etc/NEWS b/etc/NEWS
index a72be53..a88d9ec 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -179,6 +179,11 @@ questions, with a handy way to display help texts.
 +++
 ** 'switch-to-buffer-preserve-window-point' now defaults to t.
 
++++
+** The new variable 'debugger-stack-frame-as-list' allows displaying
+all call stack frames in 'backtrace' as lists.  Both debug.el and
+edebug.el have been updated accordingly.
+
 \f
 * Editing Changes in Emacs 25.2
 
diff --git a/lisp/cus-start.el b/lisp/cus-start.el
index c830ed8..c6d0ae4 100644
--- a/lisp/cus-start.el
+++ b/lisp/cus-start.el
@@ -246,6 +246,7 @@ Leaving \"Default\" unchecked is equivalent with specifying a default of
 	     (debug-ignored-errors debug (repeat (choice symbol regexp)))
 	     (debug-on-quit debug boolean)
 	     (debug-on-signal debug boolean)
+             (debugger-stack-frame-as-list debugger boolean)
 	     ;; fileio.c
 	     (delete-by-moving-to-trash auto-save boolean "23.1")
 	     (auto-save-visited-file-name auto-save boolean)
diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 22a3f39..7d27380 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -279,7 +279,9 @@ That buffer should be current already."
   (goto-char (point-min))
   (delete-region (point)
 		 (progn
-		   (search-forward "\n  debug(")
+		   (search-forward (if debugger-stack-frame-as-list
+                                       "\n  (debug "
+                                     "\n  debug("))
 		   (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index c283c16..0c2a8ad 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -3797,7 +3797,9 @@ Otherwise call `debug' normally."
 	  (forward-line 1)
 	  (delete-region last-ok-point (point)))
 
-	 ((looking-at "^  edebug")
+	 ((looking-at (if debugger-stack-frame-as-list
+                          "^  (edebug"
+                        "^  edebug"))
 	  (forward-line 1)
 	  (delete-region last-ok-point (point))
 	  )))
diff --git a/src/eval.c b/src/eval.c
index 72facd5..f07cc83 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3409,13 +3409,17 @@ Output stream used is value of `standard-output'.  */)
       else
 	{
 	  tem = backtrace_function (pdl);
+	  if (debugger_stack_frame_as_list)
+	    write_string ("(");
 	  Fprin1 (tem, Qnil);	/* This can QUIT.  */
-	  write_string ("(");
+	  if (!debugger_stack_frame_as_list)
+	    write_string ("(");
 	  {
 	    ptrdiff_t i;
 	    for (i = 0; i < backtrace_nargs (pdl); i++)
 	      {
-		if (i) write_string (" ");
+		if (i || debugger_stack_frame_as_list)
+		  write_string(" ");
 		Fprin1 (backtrace_args (pdl)[i], Qnil);
 	      }
 	  }
@@ -3838,6 +3842,10 @@ This is nil when the debugger is called under circumstances where it
 might not be safe to continue.  */);
   debugger_may_continue = 1;
 
+  DEFVAR_BOOL ("debugger-stack-frame-as-list", debugger_stack_frame_as_list,
+	       doc: /* Non-nil means display call stack frames as lists. */);
+  debugger_stack_frame_as_list = 0;
+
   DEFVAR_LISP ("debugger", Vdebugger,
 	       doc: /* Function to call to invoke debugger.
 If due to frame exit, args are `exit' and the value being returned;
-- 
2.10.0

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Eli Zaretskii]

> Date: Fri, 30 Sep 2016 12:29:56 +0200
> From: Vasilij Schneidermann <v.schneidermann@gmail.com>
> Cc: 24514@debbugs.gnu.org
> 
> > Two minor nits to make this patch perfect:
> > 
> >   . Add the variable to cus-start.el, so that it will be a defcustom
> >   . Add a NEWS entry and mention the variable in the ELisp manual
> 
> I've done these and changed the commit message to comply better with the
> changelog style.  Let me know if there's still something off.

Thanks, pushed to master.

The log entry need some tweaking to bring it to our standards; see my
commit for what was needed.

If this bug can be closed now, please do.

bug#24514: 24.5; [WIP][PATCH] Lispy backtraces

[Vasilij Schneidermann]

Feature was fully implemented and committed to the respective branch.