bug#66750: Unhelpful text in C-h v for variables with a lambda form as value

[Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Hi Alan,

> That's just your view.  You're wrong, here.  I've put little emphasis on
> bootstrap problems.

Maybe you didn't think of it explicitly while designing your solution,
but you sure do mention it very frequently in this discussion.

> That they are handled is a side effect of handling everything.

Are you saying that you're using the argument that it works in the
bootstrap case as a post-hoc rationalization of your design choices?

>> GDB is my "go to" tool when I have an error during bootstrap and I find
>> it fairly easy to use.
> OK, I found it very difficult to get a working command line for this.
> Maybe if you do it often enough it becomes second nature.

Yes, it takes some practice at first to figure out how to get the
command line, indeed.  Since it's a problem that (as a rough
approximation) people either never face or on the contrary face
regularly, that seems acceptable.

>> Note that `early-debug.el` doesn't handle "_ALL_" cases: it's only used
>> for those cases where the normal debugger can't be used.
> That's twisted reasoning.  early-debug.el will handle anything that gets
> thrown at it.

There's "handle" and there's "handle".  You seem to use "handle" to mean
that it will so what it does without crashing or returning
incorrect info.  I'd use "handle" here to mean that it does what is
expected, which it doesn't because we expect more features, which is why
we use it only in special circumstances.

>> Before catering to the same use-cases as early-debug, we should cater to
>> the use cases of the average user/developer who's never getting close to
>> the bootstrap.
> I disagree.  I don't understand why you want to split off (some of) "the
> use cases of the average user" and only handle those cases.

I don't.  Instead I'm saying we should first and foremost design the
functionality for the most common case.  And *then* see what it takes
to extend it to other cases.

> My intention is to handle _ALL_ cases, and you find this objectionable
> for some reason.

Because your focus on the fringe cases has negative impacts on the
more common case.

>> > This is the point of the defining symbol.  The "jumping" is only
>> > straightforward from a running Emacs session.  From a batch mode
>> > backtrace (e.g. from bootstrapping or ERT), it is much
>> > more cumbersome.
>> If your batch session's output is piped to an auto-reverted file in
>> compilation-mode (mine usually is, FWIW), it's small matter of making
>> sure the format is recognized.
> In other words, assume the user is advanced enough to go through all
> these motions, just to be able to work the way you do.  I suspect most
> users will not be doing this.  I certainly don't (at least, not yet).

This is only needed when the error is emitted on the stdout/stderr.
An unusual case.  Users don't have to use the above setup: they can also
open the file by hand and jump to the line by hand.  I'm only pointing
out how to make it work conveniently if you're among those rare users
who face this situation often enough that the problem you mention is one
they actually care about.

>> Still, I refuse to change the syntax of `lambda` just to make it
>> slightly easier to debug the odd bootstrap problem.
> For crying out loud, I've explained to you often enough, the syntax of
> lambda is _NOT_, I repeat _NOT_ being changed.

That's not what I see in your branch.  Now if lambda's first arg is
a symbol it means something else than what it meant earlier.

So the syntax of lambda used to

    (lambda ARGSLIST [DOCSTRING] [DECLARATIONS] [INTERACTIVEFORM] BODY...)

and now it's

    (lambda [SYMBOL] ARGSLIST [DOCSTRING] [DECLARATIONS] [INTERACTIVEFORM] BODY...)

Crying out loud or not doesn't change the fact that the above is
a change of syntax.

E.g. I see:

    +  #'(lambda
    +      byte-run--strip-list
    +            (arg)

clearly, this is using a new syntax.

> Wrong question.  I've already explained enough, I believe backtraces and
> things like C-h v should work well IN ALL CASES.  You don't.  I'm asking
> you to accept this difference in view between us.

Fine.  But please accept as well my point of view which is that your
change should not cost an extra field in every closure and should not
change the syntax of `lambda` (I don't really care about changing the
representation of interpreted functions, to be honest: I see them as
a just crutches needed for bootstrap and Edebug).

> But the information it currently prints is inadequate for lambda forms.

Agreed.

> That's what I've fixed, and it will be fixed in the "normal" backtrace
> code, too.  You don't like it, for some reason.

You got me wrong: I like the idea.
I don't like some aspects of the execution, which is why I'm suggesting
changes to address those shortcomings.

>> You could tweak Edebug to add that info, thus keeping the change localized
>> to Edebug.
> I doubt it.

I'm pretty sure it's a straightforward change :-)
[ Tho, admittedly, you may not like the place where that defining-form
symbol would appear, since I'd put it somewhere at the beginning of the body of
the function.  ]

>> Still, it changes the syntax of a very fundamental building block.
>> That's a pretty high cost in my book compared to the benefit it brings.
> You don't like my patch for some other reason.

Ah, so you're presuming that I'm arguing in bad faith, which is why
you're being so defensive.  I'm not arguing in bad faith.

The two things that I object to are the two things above (the change in
`lambda` and the extra field in closures).  I proposed ways to fix them,
but if you find some other way to fix them, that's fine by me.

> I think you're being unreasonably inflexible.  As far as I can tell,
> you haven't tried it out yet, or looked at the code, beyond
> a superficial glance at a large diff.

Trying it out did not make me see anything I didn't know (except that
the print syntax you're using will/would need changing because a list
of functions ends up as:

    ({mysym} #f(compiled-function ...) {theirsym} #f(compiled-function ...))

which looks like the list contains 4 elements instead of 2, but that's
a completely irrelevant detail at this point).

>> If you want to do that, I'd rather you keep `lambda` as before and
>> macro-expand it to a new form that has the new syntax.
> There is no new syntax, as I keep telling you.

I don't think you know what "new syntax" means, then.

>> Part of the cost of your change here is that it prevents future changes
>> that might want to use a similar syntax for other purposes.
>> IOW it "eats up namespace", tho "namespace" is not quite the right word.
> I don't know what you're talking about here.

For example, let's say that I want to extend the language such that

    (lambda x <code>)

means the same as

    (lambda (&rest x) <code>)

like it does in Scheme.  After your patch is accepted, such a change
will not be possible anymore because it conflicts with your use of the
(lambda <SYMBOL> ...) syntax.

> I think you're being thoroughly unrealistic here.  RAM in a modern
> machine is measured in gigabytes, not kilobytes.  There are around 36k
> DEFUNs/defuns/defmacros in Emacs.  Even if all were simultaneously
> loaded into RAM, that would only be around a quarter of a megabyte of
> storage used up by my patch.

I'm talking about closures.  I don't care about that extra field in
those few thousand functions.  I care about that extra field in the
closures that are dynamically generated with code such as that output by
`generator.el`.
There can easily be thousands such closures generated during the
execution of a simple command, without loading any new code.

>> Maybe in your tests, the increase is negligible, but remember: every
>> user will pay this tiny cost, and most of the users will never reap any
>> benefit from it, and those who do will only do very rarely.
> Developers will benefit.

They'll benefit just as much if you put that info in the docstring/.elc
file (such that you don't need to change the size of your bytecode
objects).  For this reason, the extra cost in unacceptable no matter how
small you think it is: it's a pure waste.

> Yes, I think of the "docstring" as the doc string.  I don't think it a
> good idea to cram other things into it which aren't the doc string.

You have to distinguish the docstring itself, and the mechanism used to
store it.  I'm talking about the internal mechanism.  I don't want to
literally place the LINE+COL info in the docstring (contrary to what
I sadly did for arglists).  Instead I want to place it in the .elc file
next to the actual docstring such that we can recover it from the very
same (FILE . OFFSET) info that points to the docstring.

IOE, I suggest we rename the "docstring" found in `.elc` at (FILE
. OFFSET) into an "introspection data" which can contain the docstring
and a few other things.


        Stefan