The poor quality of Emacs's backtraces

The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Emacs.

On running the test suite (make check) on my development version, I
recently got lots of errors and backtraces in the files.log.  This is
fair enough.

What's not good is the message that precedes (or, in the test suite
follows) the backtrace.  In the current instance, it looks like:

   Test test-kill-buffer-auto-save-delete-no condition:
    (wrong-type-argument listp
                         #[257 "\300\211\2!\262\1\207" [yes-or-no-p] 4
                               "\n\n(fn ARG124 &optional)" t])
   FAILED  376/406  test-kill-buffer-auto-save-delete-no (0.012664 sec)

So, this says that _something_ wasn't a list, without telling me what the
something was.  It may have been the value of a variable, or the result
returned from evaluating a form, but the message gives no clue.

It says "wrong-type-argument", but doesn't say why it's wrong.  It
doesn't disclose which primitive detected the fault, though Emacs could
easily do this.

It doesn't make any effort to say _where_ the bug was detected.  If
you're lucky, it might be somewhere in the first function reported in the
backtrace, but only if there aren't any frivolous condition-case's in the
code which render any remaining backtrace useless.  But even if you know
what function it's in, unless that function is short, you don't know
_where_ in that function the error has occurred.  The Emacs backtrace
functions don't output any coordinates.  This could be done easily, for
example by printing the current position and the total length for a
compiled function (something like 56/120), or by outputting neighbouring
forms for an interpreted function.  This would be helpful information for
debugging.

As already said, the actual backtrace itself is often of little use, due
to the frivolous use of condition-case's.  That's even if you stop the
test suite truncating every line at ~70 characters.  (Why is this done?)
Currently the first few lines of my backtrace look like:

    Test test-kill-buffer-auto-save-delete-no backtrace:
  {comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>)((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
  apply({comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>) (lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil)
  comp-spill-lap-function((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
  comp-spill-lap((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
  comp--native-compile((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil "/tmp/test-nativecomp-cache-Dmx7GP/30.0.50-7a56150c...")
  comp-trampoline-compile(yes-or-no-p)
  comp-subr-trampoline-install(yes-or-no-p)
  {test-overlay-regions} #f(compiled-function () #<bytecode -0x1be13de621723a1f>)()
  test-kill-buffer-auto-save(110 {test-kill-buffer-auto-save} #f(compiled-function () #<bytecode -0x1c782c52a4f77189>))
  {test-kill-buffer-auto-save} #f(compiled-function () #<bytecode 0xeb411732a4fb03f>)()

(The symbols in braces are an enhancement I'm currently working on to give
more information for anonymous functions.)

If anybody can match up the "wrong-type-argument" message to this
backtrace, they're a better hacker than me - What precisely in
comp-spill-lap-function is using some unknown list primitive on the byte
compiled function?

So, on the principle of using ALL the information that is available, we
might as well disassemble that byte-compiled function, which might give a
clue:

    \300   byte-constant yes-or-no-p
    \211   byte-dup
    \2     stack-ref 2 ; ARG124
    !      byte-call 1
    \262\1 stack-set 1
    \207   return

.... but not much of one.  The function is calling yes-or-no-p with its
parameter, and then I think it's returning yes-or-no-p's result.  Though
why it's duplicating yes-or-no-p at the top of the stack, then
overwriting it with the result is unclear.  In the test
test-kill-buffer-auto-save-delete-no, fancy things are done with cl-letf
on (symbol-function 'yes-or-no-p), so the disassembled function above
probably has something to do with that.

It's worth pointing out that there doesn't seem to be a way to get Emacs
to disassemble a function, only a symbol with a function value.

########################################################################

I will eventually track down the cause of the above bug.  But it will
have taken me MUCH longer than it would have done, had the missing
information actually been present in the backtrace.  I think I might not
be alone in wishing for these things to be improved.

So I propose that the quality of our backtraces be improved, and nominate
myself as the person to do the work.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Christopher Dimech]

> Sent: Friday, July 14, 2023 at 1:35 AM
> From: "Alan Mackenzie" <acm@muc.de>
> To: emacs-devel@gnu.org
> Subject: The poor quality of Emacs's backtraces
>
> Hello, Emacs.
>
> On running the test suite (make check) on my development version, I
> recently got lots of errors and backtraces in the files.log.  This is
> fair enough.
>
> What's not good is the message that precedes (or, in the test suite
> follows) the backtrace.  In the current instance, it looks like:
>
>    Test test-kill-buffer-auto-save-delete-no condition:
>     (wrong-type-argument listp
>                          #[257 "\300\211\2!\262\1\207" [yes-or-no-p] 4
>                                "\n\n(fn ARG124 &optional)" t])
>    FAILED  376/406  test-kill-buffer-auto-save-delete-no (0.012664 sec)
>
> So, this says that _something_ wasn't a list, without telling me what the
> something was.  It may have been the value of a variable, or the result
> returned from evaluating a form, but the message gives no clue.
>
> It says "wrong-type-argument", but doesn't say why it's wrong.  It
> doesn't disclose which primitive detected the fault, though Emacs could
> easily do this.
>
> It doesn't make any effort to say _where_ the bug was detected.  If
> you're lucky, it might be somewhere in the first function reported in the
> backtrace, but only if there aren't any frivolous condition-case's in the
> code which render any remaining backtrace useless.  But even if you know
> what function it's in, unless that function is short, you don't know
> _where_ in that function the error has occurred.  The Emacs backtrace
> functions don't output any coordinates.  This could be done easily, for
> example by printing the current position and the total length for a
> compiled function (something like 56/120), or by outputting neighbouring
> forms for an interpreted function.  This would be helpful information for
> debugging.
>
> As already said, the actual backtrace itself is often of little use, due
> to the frivolous use of condition-case's.  That's even if you stop the
> test suite truncating every line at ~70 characters.  (Why is this done?)

I quite agree that improvements to the backtrace information are needed.
It is still enormously difficult to rapidly identify the actual line of code
causing the problem from the information provided.  Most times identification
is done only because of experience than anything else - certainly not a
productive way to work.

----- Christopher Dimech
Administrator General - Naiad Informatics - Gnu Project

Society has become too quick to pass judgement and declare someone
Persona Non-Grata, the most extreme form of censure a country can
bestow.

In a new era of destructive authoritarianism, I support Richard
Stallman.  Times of great crisis are also times of great
opportunity.  I call upon you to make this struggle yours as well !

https://stallmansupport.org/
https://www.fsf.org/     https://www.gnu.org


> Currently the first few lines of my backtrace look like:
>
>     Test test-kill-buffer-auto-save-delete-no backtrace:
>   {comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>)((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
>   apply({comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>) (lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil)
>   comp-spill-lap-function((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
>   comp-spill-lap((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
>   comp--native-compile((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil "/tmp/test-nativecomp-cache-Dmx7GP/30.0.50-7a56150c...")
>   comp-trampoline-compile(yes-or-no-p)
>   comp-subr-trampoline-install(yes-or-no-p)
>   {test-overlay-regions} #f(compiled-function () #<bytecode -0x1be13de621723a1f>)()
>   test-kill-buffer-auto-save(110 {test-kill-buffer-auto-save} #f(compiled-function () #<bytecode -0x1c782c52a4f77189>))
>   {test-kill-buffer-auto-save} #f(compiled-function () #<bytecode 0xeb411732a4fb03f>)()
>
> (The symbols in braces are an enhancement I'm currently working on to give
> more information for anonymous functions.)
>
> If anybody can match up the "wrong-type-argument" message to this
> backtrace, they're a better hacker than me - What precisely in
> comp-spill-lap-function is using some unknown list primitive on the byte
> compiled function?
>
> So, on the principle of using ALL the information that is available, we
> might as well disassemble that byte-compiled function, which might give a
> clue:
>
>     \300   byte-constant yes-or-no-p
>     \211   byte-dup
>     \2     stack-ref 2 ; ARG124
>     !      byte-call 1
>     \262\1 stack-set 1
>     \207   return
>
> .... but not much of one.  The function is calling yes-or-no-p with its
> parameter, and then I think it's returning yes-or-no-p's result.  Though
> why it's duplicating yes-or-no-p at the top of the stack, then
> overwriting it with the result is unclear.  In the test
> test-kill-buffer-auto-save-delete-no, fancy things are done with cl-letf
> on (symbol-function 'yes-or-no-p), so the disassembled function above
> probably has something to do with that.
>
> It's worth pointing out that there doesn't seem to be a way to get Emacs
> to disassemble a function, only a symbol with a function value.
>
> ########################################################################
>
> I will eventually track down the cause of the above bug.  But it will
> have taken me MUCH longer than it would have done, had the missing
> information actually been present in the backtrace.  I think I might not
> be alone in wishing for these things to be improved.
>
> So I propose that the quality of our backtraces be improved, and nominate
> myself as the person to do the work.
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>
>

Re: The poor quality of Emacs's backtraces

[Mattias Engdegård]

13 juli 2023 kl. 15.35 skrev Alan Mackenzie <acm@muc.de>:

>    (wrong-type-argument listp
>                         #[257 "\300\211\2!\262\1\207" [yes-or-no-p] 4
>                               "\n\n(fn ARG124 &optional)" t])

> So, this says that _something_ wasn't a list, without telling me what the
> something was.

Actually it does. It's the element after the type, in this case the byte-compiled function #[257 ...].

> It says "wrong-type-argument", but doesn't say why it's wrong.  It
> doesn't disclose which primitive detected the fault, though Emacs could
> easily do this.

Usually it's the topmost function in the traceback.
If not, please report it as a bug. There's certainly work to be done.

> test suite truncating every line at ~70 characters.  (Why is this done?)

I agree, that's annoying. We have to truncate at some point or we'll be treated to dumps of impractical size before we know it, but 70 chars is pretty useless.

> (The symbols in braces are an enhancement I'm currently working on to give
> more information for anonymous functions.)

Keep us in touch, because it's very likely that some of us are working on the same code, with similar but different goals. Don't go it alone.

> It's worth pointing out that there doesn't seem to be a way to get Emacs
> to disassemble a function, only a symbol with a function value.

Actually there is. Just use the function `disassemble`:

(disassemble #[257 "\300\211\2!\262\1\207" [yes-or-no-p] 4 "\n\n(fn ARG124 &optional)" t])

Re: The poor quality of Emacs's backtraces

[Ihor Radchenko]

Mattias Engdegård <mattias.engdegard@gmail.com> writes:

>> test suite truncating every line at ~70 characters.  (Why is this done?)
>
> I agree, that's annoying. We have to truncate at some point or we'll be treated to dumps of impractical size before we know it, but 70 chars is pretty useless.

It would be nice if the backtraces used `pp' instead of dumb truncation.
Though would make things much, much more readable.

Of course, the `condition-case' created by ERT will still be an issue.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: The poor quality of Emacs's backtraces

[Mattias Engdegård]

14 juli 2023 kl. 10.00 skrev Ihor Radchenko <yantar92@posteo.net>:

> It would be nice if the backtraces used `pp' instead of dumb truncation.
> Though would make things much, much more readable.

Not necessarily as it also means that uninteresting data takes up much more space.
Better have a way to expand and pretty-print individual stack frames on demand in the backtrace buffer.

> Of course, the `condition-case' created by ERT will still be an issue.

Yes, and that was probably what really caused Alan's troubles. We could do something about that.

Re: The poor quality of Emacs's backtraces

[Ihor Radchenko]

Mattias Engdegård <mattias.engdegard@gmail.com> writes:

> 14 juli 2023 kl. 10.00 skrev Ihor Radchenko <yantar92@posteo.net>:
>
>> It would be nice if the backtraces used `pp' instead of dumb truncation.
>> Though would make things much, much more readable.
>
> Not necessarily as it also means that uninteresting data takes up much more space.
> Better have a way to expand and pretty-print individual stack frames on demand in the backtrace buffer.

I am a bit confused.
Aren't we talking about terminal output?

>> Of course, the `condition-case' created by ERT will still be an issue.
>
> Yes, and that was probably what really caused Alan's troubles. We could do something about that.

I think that this problem was left unresolved due to lack of ideas:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50629

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Ihor and Mattias.

On Fri, Jul 14, 2023 at 09:18:19 +0000, Ihor Radchenko wrote:
> Mattias Engdegård <mattias.engdegard@gmail.com> writes:

> > 14 juli 2023 kl. 10.00 skrev Ihor Radchenko <yantar92@posteo.net>:

[ .... ]

> I am a bit confused.
> Aren't we talking about terminal output?

> >> Of course, the `condition-case' created by ERT will still be an issue.

> > Yes, and that was probably what really caused Alan's troubles. We could do something about that.

condition-case and backtrace mix very poorly.  The problem is that, in
the event of an error, condition-case unwinds the stack _before_
executing its error handler.  So the pertinent information is discarded
before backtrace gets evaluated.

As yet, I've only got vague notions as to how to solve this.  But it
surely ought to be possible to amend condition-case, or create a variant
of condition-case, where the stack unwinding only occurs _after_
evaluating the error handler.

> I think that this problem was left unresolved due to lack of ideas:
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50629

That bug looks like just the same problems I'm having.

> -- 
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Alan Mackenzie <acm@muc.de>, emacs-devel@gnu.org
> Date: Fri, 14 Jul 2023 08:00:17 +0000
> 
> Mattias Engdegård <mattias.engdegard@gmail.com> writes:
> 
> >> test suite truncating every line at ~70 characters.  (Why is this done?)
> >
> > I agree, that's annoying. We have to truncate at some point or we'll be treated to dumps of impractical size before we know it, but 70 chars is pretty useless.
> 
> It would be nice if the backtraces used `pp' instead of dumb truncation.

That would be a definite improvement, but please don't forget that the
backtrace must be available in many situations where Emacs is
unstable.  In particular, it is completely unacceptable for a
backtrace to cause Emacs to abort, e.g., because it attempts to
display some invalid Lisp object.  (Yes, we had such problems
recently, due to the fact that the backtrace nowadays uses cl-print.)

So whatever you do to enhance the backtrace, please take care of
catching any exceptions and falling back to the good-old simple
backtrace.

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Mattias.

On Thu, Jul 13, 2023 at 16:57:52 +0200, Mattias Engdegård wrote:
> 13 juli 2023 kl. 15.35 skrev Alan Mackenzie <acm@muc.de>:

> >    (wrong-type-argument listp
> >                         #[257 "\300\211\2!\262\1\207" [yes-or-no-p] 4
> >                               "\n\n(fn ARG124 &optional)" t])

> > So, this says that _something_ wasn't a list, without telling me what the
> > something was.

> Actually it does. It's the element after the type, in this case the
> byte-compiled function #[257 ...].

You haven't grasped my point.  It doesn't tell me which variable had
or which evaluation resulted in that invalid value.  It just prints out
the value, leaving me to guess what, exactly, had that value.  Such
"guessing" can take hours or days.

> > It says "wrong-type-argument", but doesn't say why it's wrong.  It
> > doesn't disclose which primitive detected the fault, though Emacs could
> > easily do this.

> Usually it's the topmost function in the traceback.
> If not, please report it as a bug. There's certainly work to be done.

I found the bug, eventually.  I'd written (nth 5 byte-code) where what I
needed was (aref byte-code 5).  If the message had included the
information "nth" in it, I would have found the bug quickly.

> > test suite truncating every line at ~70 characters.  (Why is this done?)

> I agree, that's annoying. We have to truncate at some point or we'll
> be treated to dumps of impractical size before we know it, but 70
> chars is pretty useless.

> > (The symbols in braces are an enhancement I'm currently working on to give
> > more information for anonymous functions.)

> Keep us in touch, because it's very likely that some of us are working
> on the same code, with similar but different goals. Don't go it alone.

You're working on something similar, too?

My enhancement is to store the @dfn{defining symbol} of anonymous
functions inside the function, whether an interpreted function, or
either type of compiled function.  This symbol is the defun/defmacro
etc. within which the lambda is defined.

It's approaching the point at which it would be sensible to create a new
branch for it and commit the changes to it.

> > It's worth pointing out that there doesn't seem to be a way to get Emacs
> > to disassemble a function, only a symbol with a function value.

> Actually there is. Just use the function `disassemble`:

> (disassemble #[257 "\300\211\2!\262\1\207" [yes-or-no-p] 4 "\n\n(fn ARG124 &optional)" t])

Ah!  Thanks!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Mattias Engdegård]

14 juli 2023 kl. 12.48 skrev Alan Mackenzie <acm@muc.de>:

> It doesn't tell me which variable had
> or which evaluation resulted in that invalid value.

Here I was going to write a complaint about ERT and how it is to blame for all the world's ills, but for all its flaws it actually makes an effort to save the backtrace, using the debugger mechanism, before unwinding the stack.

(There is still plenty to fix in ERT. For example, it could use lexical closures to its advantage, and avoid these huge macro-expansions.)

The fault is perhaps closer to (my?) home: the bytecode interpreter doesn't add a stack frame for many built-in ops. For example, (cdr 3) will signal a `wrong-type-argument` error but `cdr` isn't in the backtrace.

That is something we could and should fix. At least for some ops it should be pretty easy, and even a partial remedy would be helpful. Alan, thank you for bringing this to my attention!

> My enhancement is to store the @dfn{defining symbol} of anonymous
> functions inside the function, whether an interpreted function, or
> either type of compiled function.  This symbol is the defun/defmacro
> etc. within which the lambda is defined.

Well, no, this isn't necessarily an improvement. If anything our function objects are too fat; making them more so isn't what we need.

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Mattias.

On Fri, Jul 14, 2023 at 14:35:32 +0200, Mattias Engdegård wrote:
> 14 juli 2023 kl. 12.48 skrev Alan Mackenzie <acm@muc.de>:

[ .... ]

> The fault is perhaps closer to (my?) home: the bytecode interpreter
> doesn't add a stack frame for many built-in ops. For example, (cdr 3)
> will signal a `wrong-type-argument` error but `cdr` isn't in the
> backtrace.

We can't afford to add stack frames for things like cdr.  It would make
Emacs _far_ too slow.

> That is something we could and should fix. At least for some ops it
> should be pretty easy, and even a partial remedy would be helpful.
> Alan, thank you for bringing this to my attention!

My idea for fixing this problem would be to add an extra parameter to
the CHECK_LIST (C) macro, the (symbol of) the primitive which is using
cdr.  This would be output in the backtrace error message.

There are only 1,728 occurrences of CHECK_* in the Emacs C sources.
Much of the amendment could be automated.

> > My enhancement is to store the @dfn{defining symbol} of anonymous
> > functions inside the function, whether an interpreted function, or
> > either type of compiled function.  This symbol is the defun/defmacro
> > etc. within which the lambda is defined.

> Well, no, this isn't necessarily an improvement. If anything our
> function objects are too fat; making them more so isn't what we need.

Why?  Yesterday evening, the identity of {comp-spill-lap-function} was
very helpful in locating the buggy source.

Do you have any alternative mechanism in mind for identifying anonymous
functions in backtraces?  After all, there is nothing particularly wrong
with the fatness of our function objects, provided it is not taken to
extremes - RAM is cheap and plentiful.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Mattias Engdegård]

14 juli 2023 kl. 15.07 skrev Alan Mackenzie <acm@muc.de>:

> There are only 1,728 occurrences of CHECK_* in the Emacs C sources.
> Much of the amendment could be automated.

No, we had better be careful here -- don't want to make anything slower.

> Yesterday evening, the identity of {comp-spill-lap-function} was
> very helpful in locating the buggy source.

That was yesterday. Today you wouldn't need it, because nth now appears in the backtrace (well, most of the time).

> Do you have any alternative mechanism in mind for identifying anonymous
> functions in backtraces?

I disagree with the idea of that somehow being a requirement.

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Mattias.

On Fri, Jul 14, 2023 at 20:06:02 +0200, Mattias Engdegård wrote:
> 14 juli 2023 kl. 15.07 skrev Alan Mackenzie <acm@muc.de>:

> > There are only 1,728 occurrences of CHECK_* in the Emacs C sources.
> > Much of the amendment could be automated.

> No, we had better be careful here -- don't want to make anything slower.

Aren't we always careful?  I wasn't intending to make anything slower
(except, marginally, the handling of errors).

> > Yesterday evening, the identity of {comp-spill-lap-function} was
> > very helpful in locating the buggy source.

> That was yesterday. Today you wouldn't need it, because nth now
> appears in the backtrace (well, most of the time).

That's a rather strange notion.  Whether it's "needed" or not, it's
undeniably helpful.  I think you agreed yesterday with my basic tenet,
that Emacs backtraces are of poor quality.  This is one way that quality
can be raised.

> > Do you have any alternative mechanism in mind for identifying anonymous
> > functions in backtraces?

> I disagree with the idea of that somehow being a requirement.

Why?  Are you working on anything which could remotely be considered a
competitor for this facility; something you suggested yesterday might be
the case?

I have working code implementing the putting of this extra information
into backtraces.  Again, why do you regard this as a negative feature?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Mattias Engdegård]

14 juli 2023 kl. 22.51 skrev Alan Mackenzie <acm@muc.de>:

> Aren't we always careful?  I wasn't intending to make anything slower
> (except, marginally, the handling of errors).

Your intent is not in doubt, but the execution path is so sensitive that seemingly minor changes often have outsized effect on performance. (This cuts both ways: there have also been optimisation opportunities in what appeared to be reasonably efficient code.)

Bug#64613 has been opened for this effort, and I intend to continue it shortly. I'm certainly open for ideas about how to solve the problem efficiently and effectively.

> I have working code implementing the putting of this extra information
> into backtraces.

Writing code is the easy part. What I'm wary about is rushing a solution -- your solution -- to a problem that hasn't yet been studied in sufficient depth, and with no regard to the costs involved: not just performance but also the fact that other people, myself included, will be the ones to maintain the feature for all foreseeable future.

The problem is compounded by our current suboptimal representation of closures: instead of the typical ⟨function, environment⟩ tuple, we represent them as copies of the function object with closed-over variables allocated separately and hacked into the (also copied) constant vector. Making that copied function object even bigger is the opposite of what we want to do.

There is also the question of what information should be there if any were to be added. For example, source location could arguably be more useful than function name.

But more importantly, since you said yourself that you would have been satisfied with a backtrace that included the faulting operation, and we agree that this is something we want in any case, wouldn't it make sense to begin there? The risk is much lower and it won't saddle us with irreversible maintenance costs.

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Mattias.

On Mon, Jul 17, 2023 at 17:52:12 +0200, Mattias Engdegård wrote:
> 14 juli 2023 kl. 22.51 skrev Alan Mackenzie <acm@muc.de>:

> > Aren't we always careful?  I wasn't intending to make anything slower
> > (except, marginally, the handling of errors).

> Your intent is not in doubt, but the execution path is so sensitive
> that seemingly minor changes often have outsized effect on performance.
> (This cuts both ways: there have also been optimisation opportunities
> in what appeared to be reasonably efficient code.)

Well, I've committed the code (see below).  Please actually measure it
and point out where it is actually slow, rather than pontificating on
vague abstractions.

> Bug#64613 has been opened for this effort, and I intend to continue it
> shortly. I'm certainly open for ideas about how to solve the problem
> efficiently and effectively.

> > I have working code implementing the putting of this extra
> > information into backtraces.

> Writing code is the easy part. What I'm wary about is rushing a
> solution -- your solution -- to a problem that hasn't yet been studied
> in sufficient depth, and with no regard to the costs involved: not just
> performance but also the fact that other people, myself included, will
> be the ones to maintain the feature for all foreseeable future.

Well, that's one way of ensuring nothing will ever get done.

I pushed this working code this morning to the new branch
feature/named-lambdas, precisely to give people something concrete to
study, to criticize, to measure, to improve upon, and possibly to reject.

If you're going to reject anything, please reject the code, with good
reason, not vague abstract ideas.

And please don't exaggerate the "ease" with which it was written.

> The problem is compounded by our current suboptimal representation of
> closures: instead of the typical ⟨function, environment⟩ tuple, we
> represent them as copies of the function object with closed-over
> variables allocated separately and hacked into the (also copied)
> constant vector. Making that copied function object even bigger is the
> opposite of what we want to do.

I don't agree, at least not at the moment.  The function object, all
three varieties, is big enough to hold all the information it needs to
hold.  Should it need to become marginally bigger, so be it.

> There is also the question of what information should be there if any
> were to be added. For example, source location could arguably be more
> useful than function name.

These are all things that can be changed later.  The main danger is doing
nothing but talk, talk, talk, ....  Then the topic will die down, and
come up again weeks, months in the future, possibly in a bug report, when
we will talk, talk, talk, again.  There is a cost to not fixing things.

> But more importantly, since you said yourself that you would have been
> satisfied with a backtrace that included the faulting operation, ....

I think you've misunderstood me.  That "faulting operation" is merely one
of the things I want.  I would not be satisfied by just that.  The
identity of the code referred to by each backtrace line is also
essential.

> .... and we agree that this is something we want in any case, wouldn't
> it make sense to begin there? The risk is much lower and it won't
> saddle us with irreversible maintenance costs.

Whatever we do will involve "maintenance costs".  Having coded this
change, I have some good idea of where undue maintenance costs in the
Lisp engine lie.  I don't think they're where you might think.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Ihor Radchenko]

Alan Mackenzie <acm@muc.de> writes:

> I pushed this working code this morning to the new branch
> feature/named-lambdas, precisely to give people something concrete to
> study, to criticize, to measure, to improve upon, and possibly to reject.

This is curious. I tried to branch and got several native-comp errors:

Compiling /home/yantar92/.emacs.d/straight/build/mastodon/mastodon-discover.el...

------------------ 1 ---------------------

Error: invalid-function (#<subr or>)
  mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x9b6eb28a6d796a6>))
  debug-early-backtrace()
  debug-early(error (invalid-function #<subr or>))
  or("mute")
  cl-typep("mute" or)
  {comp-cstr-intersection-homogeneous} #f(compiled-function (type) #<bytecode 0x160c2be37637e319>)(or)
  cl-some({comp-cstr-intersection-homogeneous} #f(compiled-function (type) #<bytecode 0x160c2be37637e319>) (or))
  {comp-cstr-intersection-homogeneous} #f(compiled-function (s) #<bytecode -0x14d78a424f8a3836>)(#s(comp-cstr :typeset (or) :valset nil :range nil :neg nil))
  cl-every({comp-cstr-intersection-homogeneous} #f(compiled-function (s) #<bytecode -0x14d78a424f8a3836>) (#s(comp-cstr :typeset (or) :valset nil :range nil :neg nil)))
  comp-cstr-intersection-homogeneous(#s(comp-cstr :typeset nil :valset nil :range nil :neg nil) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-cstr :typeset (or) :valset nil :range nil :neg nil))
  apply(comp-cstr-intersection-homogeneous #s(comp-cstr :typeset nil :valset nil :range nil :neg nil) (#s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-cstr :typeset (or) :valset nil :range nil :neg nil)))
  comp-cstr-intersection-no-mem(#s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-cstr :typeset (or) :valset nil :range nil :neg nil))
  comp-cstr-intersection(#s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720128370982 :slot 2) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-cstr :typeset (or) :valset nil :range nil :neg nil))
  comp-fwprop-insn((assume #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720128370982 :slot 2) (and #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-cstr :typeset (or) :valset nil :range nil :neg nil))))
  {comp-fwprop*} #f(compiled-function (--cl-var-- b) #<bytecode -0x19d7f98a5a62cd38>)(bb_10 #s(comp-block-lap :name bb_10 :insns ((assume #s(comp-mvar :typeset nil :valset (nil) :range nil :neg t :id nil :slot nil) (not #s(comp-mvar :typeset nil :valset (nil) :range nil :neg nil :id nil :slot nil))) (comment "optimized out: (assume #s(comp-mvar (t) nil nil nil 23720130134584 6) (and #s(comp-mvar (t) nil nil nil 23720129916830 6) #s(comp-mvar (t) nil nil nil nil nil)))") (comment "LAP TAG 74") (setimm #s(comp-mvar :typeset nil :valset (y-or-n-p) :range nil :neg nil :id 23720129100886 :slot 6) y-or-n-p) (comment "optimized out: (setimm #s(comp-mvar (t) nil nil nil 23720125063692 7) format)") (setimm #s(comp-mvar :typeset nil :valset ("%s this thread? ") :range nil :neg nil :id 23720125752698 :slot 8) "%s this thread? ") (comment "optimized out: (setimm #s(comp-mvar (t) nil nil nil 23720124108522 9) capitalize)") (set #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720128569294 :slot 10) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2)) (set #s(comp-mvar :typeset (string) :valset nil :range ((- . +)) :neg nil :id 23720128124550 :slot 9) (call capitalize #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720128569294 :slot 10))) (assume #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720128370982 :slot 2) (and #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2) #s(comp-cstr :typeset (or) :valset nil :range nil :neg nil))) (set #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720127813196 :slot 7) (callref format #s(comp-mvar :typeset nil :valset ("%s this thread? ") :range nil :neg nil :id 23720125752698 :slot 8) #s(comp-mvar :typeset (string) :valset nil :range ((- . +)) :neg nil :id 23720128124550 :slot 9))) (set #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720125480602 :slot 6) (callref funcall #s(comp-mvar :typeset nil :valset (y-or-n-p) :range nil :neg nil :id 23720129100886 :slot 6) #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720127813196 :slot 7))) (cond-jump #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720125480602 :slot 6) #s(comp-mvar :typeset nil :valset (nil) :range nil :neg nil :id nil :slot nil) bb_12_cstrs_0 bb_11)) :closed t :in-edges (#s(comp-edge :src #s(comp-block-lap :name bb_8 :insns ((phi #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720132853942 :slot 6) (#s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720127975326 :slot 6) bb_6) (#s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720132086474 :slot 6) bb_7)) (phi #s(comp-mvar :typeset nil :valset (base-toot-id) :range ((2 . 2)) :neg nil :id 23720132568066 :slot 5) (#s(comp-mvar :typeset nil :valset (base-toot-id) :range nil :neg nil :id 23720135050654 :slot 5) bb_6) (#s(comp-mvar :typeset nil :valset nil :range ((2 . 2)) :neg nil :id 23720129369734 :slot 5) bb_7)) (phi #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720131939414 :slot 4) (#s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720125798048 :slot 4) bb_6) (#s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720133702914 :slot 4) bb_7)) (phi #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720131516902 :slot 3) (#s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720134204414 :slot 3) bb_6) (#s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720129756462 :slot 3) bb_7)) (comment "LAP TAG 55") (setimm #s(comp-mvar :typeset nil :valset (mastodon-tl--user-in-thread-p) :range nil :neg nil :id 23720130022614 :slot 4) mastodon-tl--user-in-thread-p) (set #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720129455914 :slot 5) #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720131516902 :slot 3)) (set #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720124677586 :slot 4) (callref funcall #s(comp-mvar :typeset nil :valset (mastodon-tl--user-in-thread-p) :range nil :neg nil :id 23720130022614 :slot 4) #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720129455914 :slot 5))) (setimm #s(comp-mvar :typeset nil :valset (mastodon-http--api) :range nil :neg nil :id 23720126575366 :slot 5) mastodon-http--api) (comment "optimized out: (setimm #s(comp-mvar (t) nil nil nil 23720128149454 6) format)") (setimm #s(comp-mvar :typeset nil :valset ("statuses/%s/%s") :range nil :neg nil :id 23720131411514 :slot 7) "statuses/%s/%s") (set #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720132349698 :slot 8) #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720131516902 :slot 3)) (set #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720131514304 :slot 9) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720124801038 :slot 2)) (set #s(comp-mvar :typeset (string) :valset nil :range nil :neg nil :id 23720135345664 :slot 6) (callref format #s(comp-mvar :typeset nil :valset ("statuses/%s/%s") :range nil :neg nil :id 23720131411514 :slot 7) #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720132349698 :slot 8) #s(comp-mvar :typeset nil :valset ("mute" "unmute") :range nil :neg nil :id 23720131514304 :slot 9))) (set #s(comp-mvar :typeset (t) :valset nil :range nil :neg nil :id 23720136244128 :slot 5) (callref funcall #s(comp-mvar :typeset nil :valset (maCompiling /home/yantar92/.emacs.d/straight/build/mastodon/mastodon-client.el...

--------------------- 2 -------------------------

Error: wrong-type-argument (listp esf-every-pred)
Compiling /home/yantar92/.emacs.d/straight/build/highlight-parentheses/highlight-parentheses.el...
  mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x1e3d6368a6d7815d>))
  debug-early-backtrace()
  debug-early(error (wrong-type-argument listp esf-every-pred))
  cl--sm-macroexpand-1(#<subr macroexpand-1> #'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  apply(cl--sm-macroexpand-1 #<subr macroexpand-1> (#'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>)))))
  macroexpand-1(#'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  cl--sm-macroexpand(#'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  apply(cl--sm-macroexpand (#'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>)))))
  macroexpand(#'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  macroexp-macroexpand(#'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  macroexp--expand-all(#'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--)))))
  macroexp--all-forms((list 'function #'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--))))) 1)
  macroexp--expand-all((list 'function #'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--))))))
  macroexp--all-forms((list 'apply (list 'function #'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--))))) (list 'quote --cl-preds--) '--cl-rest--) 1)
  macroexp--expand-all((list 'apply (list 'function #'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--))))) (list 'quote --cl-preds--) '--cl-rest--))
  macroexp--all-forms((list 'lambda '(&rest --cl-rest--) (list 'apply (list 'function #'(lambda (G0 . esf-every-pred) (x) (progn (macroexp--funcall-if-compiled '{macroexp--warn-wrap} #f(compiled-function () #<bytecode 0x181d54941e9cb8cb>)) (let* ((ret nil) (--cl-var-- (symbol-value G0)) (p nil) (--cl-var-- t) --cl-var--) (while (and (consp --cl-var--) (progn (setq p (car --cl-var--)) (setq ret (funcall p x)) (if ret t (progn (setq --cl-var-- nil) (setq --cl-var-- nil))))) (setq --cl-var-- (cdr --cl-var--))) (if --cl-var-- (progn ret) --cl-var--))))) (list 'quote --cl-preds--) '--cl-rest--)) 1)
  macroexp--expand-all((lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)))
  macroexpand-all((lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)) ((:cl-symbol-macros (preds (symbol-value --cl-preds--))) (function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  {cl-symbol-macrolet} #f(compiled-function (arg1 &rest rest) "Make symbol macro definitions.\nWithin the body FORMs, references to the variable NAME will be replaced\nby EXPANSION, and (setq NAME ...) will act like (setf EXPANSION ...)." #<bytecode 0xf582c449680ab4f>)(((preds (symbol-value --cl-preds--))) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)))
  macroexpand((cl-symbol-macrolet ((preds (symbol-value --cl-preds--))) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))) ((function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  macroexp-macroexpand((cl-symbol-macrolet ((preds (symbol-value --cl-preds--))) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))) ((function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  macroexp--expand-all((cl-symbol-macrolet ((preds (symbol-value --cl-preds--))) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))))
  macroexpand-all((cl-symbol-macrolet ((preds (symbol-value --cl-preds--))) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))) ((function . cl--function-convert) (esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  {lexical-let} #f(compiled-function (bindings body) "Like `let', but lexically scoped.\nThe main visible difference is that lambdas inside BODY will create\nlexical closures as in Common Lisp." #<bytecode -0x1e658b9047a72486>)(((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)))
  macroexpand((lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))) ((esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  macroexp-macroexpand((lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))) ((esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  macroexp--expand-all((lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))))
  macroexp--all-forms({esf-every-pred} (lambda esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)))) 3)
  macroexp--expand-all(#'{esf-every-pred} (lambda esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)))))
  macroexp--all-forms((defalias 'esf-every-pred #'{esf-every-pred} (lambda esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))))) 1)
  macroexp--expand-all((defalias 'esf-every-pred #'{esf-every-pred} (lambda esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))))))
  macroexpand-all((defalias 'esf-every-pred #'{esf-every-pred} (lambda esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))))) ((esf-konstantly . {esf-konstantly} #f(compiled-function (v) #<bytecode 0x19624eb9d4f19835>)) (declare-function . byte-compile-macroexpand-declare-function) (eval-when-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode 0x1c01e2d0f45998cf>)) (eval-and-compile . {byte-compile-initial-macro-environment} #f(compiled-function (&rest body) #<bytecode -0xbc663b17c21ecc4>)) (with-suppressed-warnings . {byte-compile-initial-macro-environment} #f(compiled-function (warnings &rest body) #<bytecode 0xda6048e9c79d5>))))
  byte-compile-preprocess((defalias 'esf-every-pred #'{esf-every-pred} (lambda esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))))) t)
  #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_46>((defalias 'esf-every-pred #'{esf-every-pred} (lambda esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret))))))
  byte-compile-recurse-toplevel((defun esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)))) #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_46>)
  byte-compile-toplevel-file-form((defun esf-every-pred (&rest preds) (lexical-let ((preds preds)) (lambda (x) (loop with ret = nil for p in preds do (setq ret (funcall p x)) unless ret return nil finally return ret)))))
  #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_44>(#<buffer  *Compiler Input*>)
  byte-compile-from-buffer(#<buffer  *Compiler Input*>)
  byte-compile-file("/home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el")
  {comp-spill-lap-function} #f(compiled-function (filename) "Byte-compile FILENAME, spilling data from the byte compiler." #<bytecode 0xd740ca0141f92d4>)("/home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el")
  apply({comp-spill-lap-function} #f(compiled-function (filename) "Byte-compile FILENAME, spilling data from the byte compiler." #<bytecode 0xd740ca0141f92d4>) "/home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el" nil)
  comp-spill-lap-function("/home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el")
  comp-spill-lap("/home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el")
  comp--native-compile("/home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el" t)
  load-with-code-conversion("/tmp/emacs-async-comp-eval-sexp-fu-2WbMdM.el" "/tmp/emacs-async-comp-eval-sexp-fu-2WbMdM.el" nil t)
  command-line-1(("--eval" "(setq w32-disable-abort-dialog t)" "-l" "/tmp/emacs-async-comp-eval-sexp-fu-2WbMdM.el"))
  command-line()
  normal-top-level()
Wrong type argument: listp, esf-every-pred

----------------------- 3 --------------------
Error: wrong-type-argument (proper-list-p (mode . bug-reference-prog))
  mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0xbcddce8a6d782db>))
  debug-early-backtrace()
  debug-early(error (wrong-type-argument proper-list-p (mode . bug-reference-prog)))
  cconv-analyze-form((mode . bug-reference-prog) nil)
  cconv-analyze-form((emacs-lisp-mode (mode . bug-reference-prog) (indent-tabs-mode) (fill-column . 80)) nil)
  cconv-analyze-form(((nil (bug-reference-bug-regexp . "\\(\\b\\(?:[Ii]ssue ?#?\\|[Bb]ug ?#?\\|[Pp]atch ?#\\|RFE ?#\\|PR [a-z+-]+/\\)\\([0-9]+\\(?:#[0-9]+\\)?\\)\\)") (bug-reference-url-format . "https://github.com/emacs-helm/helm/issues/%s") (byte-compile-warnings not obsolete docstrings docstrings-non-ascii-quotes)) (emacs-lisp-mode (mode . bug-reference-prog) (indent-tabs-mode) (fill-column . 80))) nil)
  cconv-closure-convert(((nil (bug-reference-bug-regexp . "\\(\\b\\(?:[Ii]ssue ?#?\\|[Bb]ug ?#?\\|[Pp]atch ?#\\|RFE ?#\\|PR [a-z+-]+/\\)\\([0-9]+\\(?:#[0-9]+\\)?\\)\\)") (bug-reference-url-format . "https://github.com/emacs-helm/helm/issues/%s") (byte-compile-warnings not obsolete docstrings docstrings-non-ascii-quotes)) (emacs-lisp-mode (mode . bug-reference-prog) (indent-tabs-mode) (fill-column . 80))) nil)
  byte-compile-preprocess(((nil (bug-reference-bug-regexp . "\\(\\b\\(?:[Ii]ssue ?#?\\|[Bb]ug ?#?\\|[Pp]atch ?#\\|RFE ?#\\|PR [a-z+-]+/\\)\\([0-9]+\\(?:#[0-9]+\\)?\\)\\)") (bug-reference-url-format . "https://github.com/emacs-helm/helm/issues/%s") (byte-compile-warnings not obsolete docstrings docstrings-non-ascii-quotes)) (emacs-lisp-mode (mode . bug-reference-prog) (indent-tabs-mode) (fill-column . 80))) t)
  #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_46>(((nil (bug-reference-bug-regexp . "\\(\\b\\(?:[Ii]ssue ?#?\\|[Bb]ug ?#?\\|[Pp]atch ?#\\|RFE ?#\\|PR [a-z+-]+/\\)\\([0-9]+\\(?:#[0-9]+\\)?\\)\\)") (bug-reference-url-format . "https://github.com/emacs-helm/helm/issues/%s") (byte-compile-warnings not obsolete docstrings docstrings-non-ascii-quotes)) (emacs-lisp-mode (mode . bug-reference-prog) (indent-tabs-mode) (fill-column . 80))))
  byte-compile-recurse-toplevel(((nil (bug-reference-bug-regexp . "\\(\\b\\(?:[Ii]ssue ?#?\\|[Bb]ug ?#?\\|[Pp]atch ?#\\|RFE ?#\\|PR [a-z+-]+/\\)\\([0-9]+\\(?:#[0-9]+\\)?\\)\\)") (bug-reference-url-format . "https://github.com/emacs-helm/helm/issues/%s") (byte-compile-warnings not obsolete docstrings docstrings-non-ascii-quotes)) (emacs-lisp-mode (mode . bug-reference-prog) (indent-tabs-mode) (fill-column . 80))) #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_46>)
  byte-compile-toplevel-file-form(((nil (bug-reference-bug-regexp . "\\(\\b\\(?:[Ii]ssue ?#?\\|[Bb]ug ?#?\\|[Pp]atch ?#\\|RFE ?#\\|PR [a-z+-]+/\\)\\([0-9]+\\(?:#[0-9]+\\)?\\)\\)") (bug-reference-url-format . "https://github.com/emacs-helm/helm/issues/%s") (byte-compile-warnings not obsolete docstrings docstrings-non-ascii-quotes)) (emacs-lisp-mode (mode . bug-reference-prog) (indent-tabs-mode) (fill-column . 80))))
  #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_44>(#<buffer  *Compiler Input*>)
  byte-compile-from-buffer(#<buffer  *Compiler Input*>)
  byte-compile-file("/home/yantar92/.emacs.d/straight/build/helm/.dir-locals.el")
  {comp-spill-lap-function} #f(compiled-function (filename) "Byte-compile FILENAME, spilling data from the byte compiler." #<bytecode 0xf3bf1a26c9ad90a>)("/home/yantar92/.emacs.d/straight/build/helm/.dir-locals.el")
  apply({comp-spill-lap-function} #f(compiled-function (filename) "Byte-compile FILENAME, spilling data from the byte compiler." #<bytecode 0xf3bf1a26c9ad90a>) "/home/yantar92/.emacs.d/straight/build/helm/.dir-locals.el" nil)
  comp-spill-lap-function("/home/yantar92/.emacs.d/straight/build/helm/.dir-locals.el")
  comp-spill-lap("/home/yantar92/.emacs.d/straight/build/helm/.dir-locals.el")
  comp--native-compile("/home/yantar92/.emacs.d/straight/build/helm/.dir-locals.el" nil)
  load-with-code-conversion("/tmp/emacs-async-comp-.dir-locals-lrWjJO.el" "/tmp/emacs-async-comp-.dir-locals-lrWjJO.el" nil t)
  command-line-1(("--eval" "(setq w32-disable-abort-dialog t)" "-l" "/tmp/emacs-async-comp-.dir-locals-lrWjJO.el"))
  command-line()
  normal-top-level()
Wrong type argument: proper-list-p, (#<symbol mode at 445> . #<symbol bug-reference-prog at 452>)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Ihor.

On Mon, Jul 17, 2023 at 19:50:50 +0000, Ihor Radchenko wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > I pushed this working code this morning to the new branch
> > feature/named-lambdas, precisely to give people something concrete to
> > study, to criticize, to measure, to improve upon, and possibly to reject.

> This is curious. I tried to branch and got several native-comp errors:

Thanks for actually trying this out!

One thing I forgot to say: you need to run make bootstrap, since the
format of some internal structures is being changed.  If you didn't do
this, please try it.  If you did run make bootstrap, please let me know,
then I will take a closer look at your backtraces.  Thanks!

> Compiling /home/yantar92/.emacs.d/straight/build/mastodon/mastodon-discover.el...

[ .... ]

> -- 
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Ihor Radchenko]

Alan Mackenzie <acm@muc.de> writes:

>> This is curious. I tried to branch and got several native-comp errors:
>
> Thanks for actually trying this out!
>
> One thing I forgot to say: you need to run make bootstrap, since the
> format of some internal structures is being changed.  If you didn't do
> this, please try it.  If you did run make bootstrap, please let me know,
> then I will take a closer look at your backtraces.  Thanks!

I did. Now, to be sure, I also did make extraclean; ./autogen...;
./configure...; make boostrap
And then also removed eln-cache dir.

Things changed somewhat:

In toplevel form:
eval-sexp-fu.el:121:2: Warning: Package cl is deprecated
eval-sexp-fu.el:145:2: Warning: custom-declare-variable `eval-sexp-fu-flash-error-face' docstring wider than 80 characters
/home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el: Error: Wrong type argument listp

/home/yantar92/.emacs.d/straight/build/helm-dash/helm-dash.el: Error: Wrong type argument listp

No backtrace this time.
Not sure if it is related to the branch.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Ihor.

On Tue, Jul 18, 2023 at 11:54:36 +0000, Ihor Radchenko wrote:
> Alan Mackenzie <acm@muc.de> writes:

> >> This is curious. I tried to branch and got several native-comp errors:

> > Thanks for actually trying this out!

> > One thing I forgot to say: you need to run make bootstrap, since the
> > format of some internal structures is being changed.  If you didn't do
> > this, please try it.  If you did run make bootstrap, please let me know,
> > then I will take a closer look at your backtraces.  Thanks!

> I did. Now, to be sure, I also did make extraclean; ./autogen...;
> ./configure...; make boostrap
> And then also removed eln-cache dir.

> Things changed somewhat:

> In toplevel form:
> eval-sexp-fu.el:121:2: Warning: Package cl is deprecated
> eval-sexp-fu.el:145:2: Warning: custom-declare-variable `eval-sexp-fu-flash-error-face' docstring wider than 80 characters
> /home/yantar92/.emacs.d/straight/build/eval-sexp-fu/eval-sexp-fu.el: Error: Wrong type argument listp

> /home/yantar92/.emacs.d/straight/build/helm-dash/helm-dash.el: Error: Wrong type argument listp

> No backtrace this time.

Excellent!

> Not sure if it is related to the branch.

I've not been able to install helm successfully on standard Emacs, but
those "error messages" illustrate to some extent the problem I'm trying
to solve.  They give no location, beyond a file name, for the problem;
they don't say what isn't a list, and don't say what primitive detected
its not being a list.

If you know helm well, can you say whether or not it handles function
structure explicitly?  Doing things like assuming the element of a
lambda form following 'lambda is the parameter list?  This might be the
cause of the problem.

> -- 
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Ihor Radchenko]

Alan Mackenzie <acm@muc.de> writes:

> If you know helm well, can you say whether or not it handles function
> structure explicitly?  Doing things like assuming the element of a
> lambda form following 'lambda is the parameter list?  This might be the
> cause of the problem.

No idea, sorry.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: The poor quality of Emacs's backtraces

[Mattias Engdegård]

17 juli 2023 kl. 21.02 skrev Alan Mackenzie <acm@muc.de>:

> Well, I've committed the code (see below).  Please actually measure it
> and point out where it is actually slow,

That is your obligation, not ours. You cannot demand that other people accept your assertion that it is fast or have to prove otherwise. Measuring performance properly is actual work.

> And please don't exaggerate the "ease" with which it was written.

I'm sure it was most taxing work.

> I don't agree, at least not at the moment.  The function object, all
> three varieties, is big enough to hold all the information it needs to
> hold.  Should it need to become marginally bigger, so be it.

It's apparently not big enough since you had to enlarge it.

Now please remember that the current closure problem is not your fault: it's an unsatisfactory set of data structures that we would like to clean up. But it is what we have to work with for the time being, and it's unclear when we will be able to do something about it (it's a considerably bigger task).

> These are all things that can be changed later.  The main danger is doing
> nothing but talk, talk, talk, ....

This kind of argument is unhelpful. Implying that there is an urgency and that we therefore should accept your solution without delay does not go down well. Nor does attempts to brush away valid criticism as things that can be changed later.

There is no crisis, and rushing through changes will be of benefit to nobody in the long run.

> That "faulting operation" is merely one
> of the things I want.  I would not be satisfied by just that.  The
> identity of the code referred to by each backtrace line is also
> essential.

There is a difference between what you personally want, and what you reasonably can expect to impose upon others.

> Whatever we do will involve "maintenance costs".

You misunderstand. I'm talking about adding a feature which we cannot easily undo or even change once it's in place. Whenever we do that, we will have to be certain that it's the right one.

Re: The poor quality of Emacs's backtraces

[Alan Mackenzie]

Hello, Mattias.

On Wed, Jul 19, 2023 at 12:33:51 +0200, Mattias Engdegård wrote:
> 17 juli 2023 kl. 21.02 skrev Alan Mackenzie <acm@muc.de>:

> > Well, I've committed the code (see below).  Please actually measure it
> > and point out where it is actually slow,

> That is your obligation, not ours. You cannot demand that other people
> accept your assertion that it is fast or have to prove otherwise.
> Measuring performance properly is actual work.

You've distorted my point by inappropriate snipping of the context.

> > And please don't exaggerate the "ease" with which it was written.

> I'm sure it was most taxing work.

> > I don't agree, at least not at the moment.  The function object, all
> > three varieties, is big enough to hold all the information it needs
> > to hold.  Should it need to become marginally bigger, so be it.

> It's apparently not big enough since you had to enlarge it.

You've misunderstood the English usage, here.  "is big enough to hold
all the information ...." is talking about the design principle.  We
don't decide how big a function object should be, then see how much
information we can pack into this.

> Now please remember that the current closure problem is not your
> fault: it's an unsatisfactory set of data structures that we would
> like to clean up. But it is what we have to work with for the time
> being, and it's unclear when we will be able to do something about it
> (it's a considerably bigger task).

What closure problem?  Has this been expounded in a post on emacs-devel?
(A reference would be appreciated if so).  Who is the "we" in this
context?  Why do you think it will be such a big task to fix it?

> > These are all things that can be changed later.  The main danger is doing
> > nothing but talk, talk, talk, ....

> This kind of argument is unhelpful. Implying that there is an urgency
> and that we therefore should accept your solution without delay does
> not go down well. Nor does attempts to brush away valid criticism as
> things that can be changed later.

> There is no crisis, and rushing through changes will be of benefit to
> nobody in the long run.

Note that nobody in this thread has countered my argument with "I find
the Emacs backtrace perfectly satisfactory, and don't think any change
is needed.".

> > That "faulting operation" is merely one
> > of the things I want.  I would not be satisfied by just that.  The
> > identity of the code referred to by each backtrace line is also
> > essential.

> There is a difference between what you personally want, and what you
> reasonably can expect to impose upon others.

I get the feeling I am being enticed into a fight with you.  You've
replied to things in my posts, giving them meanings and nuances that
they weren't intended to carry in the first place, partly by selective
snipping of the context.

I don't want to play this game with you.  I've produced working code,
you have given no sign that you've actually tried it, or looked at it,
yet you don't seem short of disparagement for it.

> > Whatever we do will involve "maintenance costs".

> You misunderstand. I'm talking about adding a feature which we cannot
> easily undo or even change once it's in place. Whenever we do that, we
> will have to be certain that it's the right one.

That involves looking at it and trying it out.

This will be my last post in this arm of the thread.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The poor quality of Emacs's backtraces

[Michael Heerdegen]

Alan Mackenzie <acm@muc.de> writes:

>     Test test-kill-buffer-auto-save-delete-no backtrace:
>   {comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>)((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
>   apply({comp-spill-lap-function} #f(compiled-function (form) "Byte-compile FORM, spilling data from the byte compiler." #<bytecode 0xa35cf62c052f23d>) (lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil)
>   comp-spill-lap-function((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
>   comp-spill-lap((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))
>   comp--native-compile((lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))) nil "/tmp/test-nativecomp-cache-Dmx7GP/30.0.50-7a56150c...")
>   comp-trampoline-compile(yes-or-no-p)

So what is it what errors?  Just

(comp-spill-lap-function
 '(lambda (arg124 &optional) (let ((f #'yes-or-no-p)) (funcall f arg124))))

?  Does evaluating that work in your session?  Maybe load the source of
`comp-spill-lap-function' before running the test to get a better
backtrace (or just eval the expression in the test to get a real debugger).

> [...]
>     \300   byte-constant yes-or-no-p
>     \211   byte-dup
>     \2     stack-ref 2 ; ARG124
>     !      byte-call 1
>     \262\1 stack-set 1
>     \207   return

Isn't that just the above lambda to be compiled (and is that function
called at all)?


Michael.