bug#66750: Unhelpful text in C-h v for variables with a lambda form as value

[Alan Mackenzie <acm@muc.de>]

Hello, Stefan.

On Wed, Nov 01, 2023 at 08:47:30 -0400, Stefan Monnier wrote:
> Alan Mackenzie [2023-10-30 09:44:54] wrote:

[ .... ]

> Thanks for putting into words how I feel :-)

:-)

> > Not yet.  Putting complicated stuff into backtrace code is a bad idea -
> > look what happened when cl-print got put into debug-early.el.  Bugs.

> Bootstrap problems are rare, and difficult to predict, so we fix them in
> a reactive way.  Debugging a bootstrap is generally more difficult.

I'm fully aware of this, having spent a significant part of the last few
years doing just that.  I aim to make it easier, as I have already done
to a certain degree.

> That's also true when trying to debug GDB (the usual answer is to use
> another GDB, and in Emacs you can do the same: debug from another
> debugger, such as GDB, rather than from the built-in debugger).

That doesn't seem relevant to bootstrapping bugs.  Using gdb on part of
the bootstrap is quite difficult.  Using edebug is not possible, as far
as I'm aware.

> We shouldn't design the featureset for the corner case of when the
> bootstrap fails, because it is a very small fraction of the use-cases,
> that 99.99% of Emacs users never face.

We should design the feature set to handle _ALL_ cases, including that
of bootstrapping.  That's why I wrote debug-early.el, for example; it's
proved its worth, certainly for me personally.

> >> > Forcing a user to look somewhere else for pertinent information can
> >> > only increase that level of stress.
> >> Any concrete evidence to back this claim?
> > Of course not.  Just as you've got no evidence to back up the contrary.

> Fist, I did not make a claim to the contrary, so the burden of proof is
> not on me.  This said, I believe there is ample evidence out there,
> because all the programming tools I've ever used try first and foremost
> to let you jump to the corresponding source code.  The source code is
> what programmers are familiar with, so being able to jump from the
> backtrace to the corresponding source code is generally the best known
> way to help the user find the pertinent info.

Even better is if the relevant information is present on the screen or
in the backtrace without having to jump to the source code.  This is the
point of the defining symbol.  The "jumping" is only straightforward
from a running Emacs session.  From a batch mode backtrace (e.g. from
bootstrapping or ERT), it is much more cumbersome.

> >From the FILE+LINE+COL info the user can recover the info your patch
> provides, but the reverse is generally not true.  Of course, there is
> the exception of when the source code has been changed or is missing or
> somesuch, but I don't think we should consider those cases when
> designing the featureset either, because here as well, this is expected
> to be a fringe case.

I would expect it to be quite common.  We clearly differ in our views of
what backtraces should do.  You seem to want them only to cover the most
usual cases, I want it to cover all cases, or as many as possible when
all is not possible.  Also, I think the backtrace code should be as
robust as possible, you don't seem to.

Anyhow, I now see you are correct in saying that we need file and
position information.  I've scanned the sources for uses of lambda at
the top level, and there are functions such as mapc, define-key, ...
which have no symbol associated with them that could be used as the
defining symbol.

I will be working on putting this file and position information into the
code.  It will be tedious rather than difficult.

[ .... ]

> > So with all the extra complexity you want to add, you'll risk getting a
> > recursive error and recursive backtrace.

> I want the better info, even if that means a slight increase in
> complexity and the associated risk that I have to go and fix a bug in
> that code next year, yes.

Debugging tools are a special case: if they stop working, the normal
means of diagnosing bugs don't work on them.  Thus they should be
robust.

[ .... ]

> > Which looks like you intend to strip it [debugging instrumentation]
> > out, leaving interpreted code as a special case without the
> > instrumentation.

> Yes.

OK, we have to disagree on this.  For example typing d in edebug to get
a backtrace should, in my view, show that info in the backtrace.  Your
way would make it unavailable.

> >> So it'll always be a best effort, where we try and cover the
> >> important cases.
> > Which is a good deal better than the current state of affairs.

> But as it currently stands, it comes at some costs:
> - changing the syntax of `lambda`.

lambda in the bulk of the sources remains unchanged.  Only for very
occasional cases has the syntax been enhanced.

> - changing the representation of functions (both bytecode and interpreted).

Don't forget the native compiled, too!  Only the change to interpreted
code is non-trivial.  For the compiled code, there has simply been an
extra field inserted at or near the end of the function structures.

> - added runtime costs even when the feature unused (no `C-h v`, no
>   backtraces), which is the most likely use-case.

There are no added runtime costs beyond the unmeasurably minute.  I've
compared before and after timings, and the differences, such as they
were, were much less than 1% and swamped by the random variation in
timings anyway.

> Storing the info in the docstring would fix the second and third point.
> Using the FILE+LINE+COL info would fix the first.

I don't know what you mean by "docstring".  You could mean either in the
..elc/.eln file or in the function structure or both.  I don't see
docstring storage as a very good idea.  It seems like a kludge, and
would complicate Emacs's handling, much as the existing new code
complicates it.

> >> > I still believe that the defining symbol scheme is the most practical
> >> > way of conveying the needed extra information to the user.
> >> Obviously.
> >> > Otherwise I would have hacked something different.  ;-)
> >> I don't think that's how it works, sadly.
> > What are you talking about with "that"?

> I have the impression that you're defending your design because that's
> what you've coded.  :-(

Only partly.  Mainly it was because I thought these things out at an
early stage of development.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).