bug#38173: describe-variable: Also tell user *where* variable was changed

bug#38173: describe-variable: Also tell user *where* variable was changed

[積丹尼 Dan Jacobson]

^H v w3m-command

   w3m-command is a variable defined in ‘w3m.el’.
   Its value is "/usr/bin/w3m"
   Original value was nil

User thinks:
Hmmm, it was changed I see.
Where did the change happen?
Did it get changed in w3m.el? But that's where it was first defined. So
it wouldn't have happened there.
He looks in .emacs . Wait, he doesn't even have a .emacs.
He gives up.

So you only tell him half the story.
Either record at least the last place a variable was changed.
Or don't tell him it was changed.

emacs-version "26.3" .

Yes, if he changed in by hand in his *scratch* buffer etc., even say that too.

bug#38173: describe-variable: Also tell user *where* variable was changed

[Katsumi Yamaoka]

On Tue, 12 Nov 2019 09:14:58 +0800, 積丹尼 Dan Jacobson wrote:
> ^H v w3m-command
>    w3m-command is a variable defined in ‘w3m.el’.
>    Its value is "/usr/bin/w3m"
>    Original value was nil

Please read the doc string for the reason why it is changed.

> User thinks:
> Hmmm, it was changed I see.
> Where did the change happen?
> Did it get changed in w3m.el? But that's where it was first defined. So
> it wouldn't have happened there.

w3m.el does it.  I don't know why the original author designed
it so to be changed from the default nil to a reasonable value
instead of making it default to it.  Anyway it will be changed
if and only if it was nil initially; you can set it as you like.

> He looks in .emacs . Wait, he doesn't even have a .emacs.
> He gives up.

> So you only tell him half the story.
> Either record at least the last place a variable was changed.
> Or don't tell him it was changed.

> emacs-version "26.3" .

> Yes, if he changed in by hand in his *scratch* buffer etc., even say that too.

I think it is very useful for debugging if I can see what changed
it and when it was changed, though I also think it would cost.

bug#38173: describe-variable: Also tell user *where* variable was changed

[積丹尼 Dan Jacobson]

OK, here is another example

    read-buffer-completion-ignore-case is a variable defined in ‘C source code’.
    Its value is t
    Original value was nil

It's like telling users "you transferred $123 from your bank account"
but not telling users who they transferred it to.

bug#38173: describe-variable: Also tell user *where* variable was changed

[Phil Sainty]

On 2019-11-12 15:45, 積丹尼 Dan Jacobson wrote:
> It's like telling users "you transferred $123 from your bank
> account" but not telling users who they transferred it to.

That's pretty crucial information for a bank transfer.  Not so much
for every arbitrary variable change (and I think you'd need to be
making an incredible number of bank transactions for the analogy to
hold up :)

Logging every change of every variable made during the execution of a
computer program might carry a noticeable cost, vs the absolutely
tiny number of cases where the logged information would ever be
inspected, so it's not obvious that this would be worthwhile on the
whole?

However, since 26.1 you can do this yourself on a case-by-case basis
when you wish to investigate some particular situation. See: C-h f
add-variable-watcher and remove-variable-watcher, and commands M-x
[cancel-]debug-on-variable-change

Some extremely basic logging without invoking the debugger might be
something like:

     (defun var-watcher-backtrace (symbol newval operation where)
       "Used with `add-variable-watcher' to log details to *Messages*."
       (let ((frames (backtrace-frames)))
         (message "%S %S %S %S %S" symbol newval operation where
                  (pp-to-string frames))))

     (add-variable-watcher 'foo 'var-watcher-backtrace)


I suppose the variable watcher mechanism could check another variable
to decide whether to do the sort of "global watching" being requested
here.  Then users could turn the behaviour on if desired.


-Phil

bug#38173: describe-variable: Also tell user *where* variable was changed

[積丹尼 Dan Jacobson]

(OK, it's like telling the user your current balance is $0, your original
balance is $1000, and throwing out even the final record of what happened.
At least don't tell him his original balance then.)

Gasp, that var-watcher stuff is so complicated.

How about:
At the beginning of the user's .emacs:
(setq tracked-variables LIST_OF_TRACKED_VARIABLES)

And then if he later does describe-variable on one of them,
he gets all the places and values that variable was set.

The tracked-variables docsrting should warn: this is for debugging use
only, and adds a burden...

bug#38173: describe-variable: Also tell user *where* variable was changed

[Phil Sainty]

On 13/11/19 12:28 PM, 積丹尼 Dan Jacobson wrote:
> Gasp, that var-watcher stuff is so complicated.

Which bit?

If you've unfamiliar with elisp then it possibly seems complicated,
but if/when you learn some more about the language then I think you'll
find that it's a very straightforward piece of code.

The only bit that wasn't directly derived from the documentation at
C-h f add-variable-watcher was the use of `backtrace-frames' (and
pretty-printing that with `pp-to-string').

And the M-x debug-on-variable-change command isn't complicated at
all, and would quite likely show you what you want to see.  You just
need to know debugger command 'c' to continue, 'q' to quit, and the
M-x cancel-debug-on-variable-change command when you're all done.


> How about:
> At the beginning of the user's .emacs:
> (setq tracked-variables LIST_OF_TRACKED_VARIABLES)

Well for specific lists you could do something along the lines
that I suggested, just for all variables in your list.

I'm somewhat inclined to suggest that IF something like this was
done, a global switch would make more sense.

It still might not have the effect you wanted, though -- it's
possible to change the apparent / user-facing value of some variables
without changing the *actual* value of the variable at all.  This is
because of the internal structure of lists in lisp.  A list variable
points at a cons cell, and so long as it remains pointing at a given
cons the variable has the same value; but this doesn't prevent the
list (which is a chain of cons cells) from being altered.  e.g.

(defvar foo '(a b c))
=> foo
foo
=> (a b c)
(debug-on-variable-change 'foo)
=> nil
(setcar foo 'd)
=> d [no debugger was invoked]
foo
=> (d b c)
(setq foo (cdr foo))
Debugger entered--setting foo to (b c):
  (debug--implement-debug-watch foo (b c) set nil)
  (setq foo (cdr foo))
  ...

If you have some reference (maybe even a different variable) pointing
into that list, you can change the list without Emacs ever seeing a
change to the original variable.

(This doesn't always happen, but it certainly *can* happen, and it
might lead to you thinking that the proposed feature wasn't working,
even if it was working as well as it could be expected to.)

I think it's an interesting idea, which would be pretty neat if it
could be made to work; but I don't know if it's practical, and I'm
dubious that it could be comprehensive enough for your liking.


-Phil

bug#38173: describe-variable: Also tell user *where* variable was changed

[積丹尼 Dan Jacobson]

This all has to be at the knowledge level of the user who has put his
first setq in his first .emacs file.

He uses describe-variable and is told some 3rd party has changed the
variable too and wants to know who.

PS> I'm somewhat inclined to suggest that IF something like this was
PS> done, a global switch would make more sense.

Indeed, the describe-variable docstring could mention: "if you want more
details, set "global-variable-tracking" to t, and restart
emacs. But note it will slow down emacs, so only use for debugging."

PS> It still might not have the effect you wanted, though -- it's
PS> possible to change the apparent / user-facing value of some variables
PS> without changing the *actual* value of the variable at all.  This is
PS> because of the internal structure of lists in lisp...

Sounds like a security / coverup risk. Maybe with the expensive
global-variable-tracking turned on, describe-variable could double check
for such tampering upon looking up a variable.

bug#38173: describe-variable: Also tell user *where* variable was changed

[Phil Sainty]

On 2019-11-14 04:06, 積丹尼 Dan Jacobson wrote:
> PS> It still might not have the effect you wanted, though -- it's
> PS> possible to change the apparent / user-facing value of some 
> variables
> PS> without changing the *actual* value of the variable at all.  This 
> is
> PS> because of the internal structure of lists in lisp...
> 
> Sounds like a security / coverup risk. Maybe with the expensive
> global-variable-tracking turned on, describe-variable could double 
> check
> for such tampering upon looking up a variable.

It's not "tampering".  It's just the fundamental nature of lists in lisp
(all dialects of lisp, AFAIK).  You're only seeing it as a concern in 
the
context of a feature which doesn't exist.

It would, I think, be *dramatically* (perhaps prohibitively) more 
expensive
to perform that kind of checking.  This could no longer be done by the
variable-watcher mechanism.  In principle, every time ANY lisp object 
was
modified at all, Emacs would need to check to see whether the object 
(pre-
modification) was a value -- or a nested sub-component, to any arbitrary
depth, of a value -- of ANY variable in the system, so that it could 
register
that the change affected the user-facing value of that variable.


-Phil

bug#38173: describe-variable: Also tell user *where* variable was changed

[Lars Ingebrigtsen]

Phil Sainty <psainty@orcon.net.nz> writes:

> It would, I think, be *dramatically* (perhaps prohibitively) more
> expensive to perform that kind of checking.

Yes, I think it's unrealistic (to put it mildly) to add something like
this to Emacs, so I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no