How to debug modification to a variable value?

How to debug modification to a variable value?

[Tassilo Horn]

Hi all,

today I had some strange problems with a major mode we use at my
institute, and that I created and maintain.  This mode defines some
buffer-local variable:

--8<---------------cut here---------------start------------->8---
(defvar tg-schema-alist nil
  "The schema of the current TG file.")
(make-variable-buffer-local 'tg-schema-alist)
--8<---------------cut here---------------end--------------->8---

This variable is set once, when a new graph is loaded (whatever that
means, but there is only one function that setqs the variable).

Also, there's a function, which refreshes some font-locking stuff when
saving.  This function is added to `after-save-hook' locally.

--8<---------------cut here---------------start------------->8---
after-save-hook is a variable defined in `files.el'.
Its value is 
(t greql-set-fontlock-types-regex)

Local in buffer method-calls-per-activity.greql; global value is nil
--8<---------------cut here---------------end--------------->8---

Now, my problem was that under some circumstances after saving, the
buffer-local value of `tg-schema-alist' was gone, i.e. set to nil.

I double-checked `greql-set-fontlock-types-regex' that it doesn't modify
`tg-schema-alist', and it doesn't.  I also removed all destructive
function calls in there, although it operates only on a list created by
`mapcar', and that's a copy anyway, right?

Till now, I couldn't reproduce this behavior.  After restarting emacs,
the problem was gone first.  Suddenly, it appeared again.  Maybe it has
something to do with using the same buffer in many frames (tty and X),
but I cannot tell for sure...

So what I need is some way to be put in the debugger when the value of
`tg-schema-alist' is modified.  Is that feasible?  I tried adding an
after advice to `setq' which does exactly that, but that screwed my
emacs instance.  I guess it's no good idea to advice such primitives...

Bye,
Tassilo

RE: How to debug modification to a variable value?

[Drew Adams]

> I also removed all destructive
> function calls in there, although it operates only on a list 
> created by `mapcar', and that's a copy anyway, right?

It's a top-level copy only. That is, each element in the original list is
processed by the function arg, and the results are consed up to produce a new
list.

If, for example, the original list elements are themselves lists, then,
depending on the function that is passed to mapcar, the original elements or
parts of them might well be included as elements of the final list, and their
own list structure might have been modified by the function arg.

The only new conses you can be sure about are those created by mapcar itself, as
it conses up the results. Take a close look at the function you pass to mapcar.
If it is destructive, then that could be where your problem lies.

Re: How to debug modification to a variable value?

[Davis Herring]

> Now, my problem was that under some circumstances after saving, the
> buffer-local value of `tg-schema-alist' was gone, i.e. set to nil.

There's a difference between "gone" and "set to nil", even for
automatically-buffer-local variables.  Does it still have a buffer-local
value (which is wrong), or is it using the default (again)?  C-h v will
say.

> I double-checked `greql-set-fontlock-types-regex' that it doesn't modify
> `tg-schema-alist', and it doesn't.  I also removed all destructive
> function calls in there, although it operates only on a list created by
> `mapcar', and that's a copy anyway, right?

Destructive operations on lists can't set the value of a variable to nil,
nor can they invoke `kill-local-variable'.  `set[qf]?' is of course
destructive, as are `kill-local-variable' and `kill-all-local-variables';
have you looked for all of those?

> So what I need is some way to be put in the debugger when the value of
> `tg-schema-alist' is modified.  Is that feasible?  I tried adding an
> after advice to `setq' which does exactly that, but that screwed my
> emacs instance.  I guess it's no good idea to advice such primitives...

I don't that you can do this in the Emacs debugger, but you should be able
to use a watchpoint in gdb for this purpose.  Find where the buffer-local
value of the variable is stored, set the watchpoint, and then save.

Meanwhile, my psychic powers suggest that you have a `let' binding of the
variable in question and switch buffers within the `let', or else that
something is reasserting your major mode so that
`kill-all-local-variables' is getting called.  You could test for the
latter with the `permanent-local' property.

Davis

PS - Sorry if some of these instructions are obvious; I don't know just
what you know!

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: How to debug modification to a variable value?

[Tassilo Horn]

"Drew Adams" <drew.adams@oracle.com> writes:

>> I also removed all destructive function calls in there, although it
>> operates only on a list created by `mapcar', and that's a copy
>> anyway, right?
>
> It's a top-level copy only. That is, each element in the original list
> is processed by the function arg, and the results are consed up to
> produce a new list.

I see.  But the only destructive functions I used are `nconc' and `delq'
(to filter nil values), and those operate on a list produced with `cons'
and `list' only on basic values.  And when the problem appeared, I
changed those calls to `append' and `remove' and re-evaled the function
definition (*), but the problem persisted.

And another thing which is a bit puzzling: I used that mode for month,
and till yesterday it never occured.  If it was a problem with
destructive functions, shouldn't it occur every time?

The main question still stands: is it possible to debug the modification
of a variable's value?  That would also find the place in the code where
a destructive function causes harm.

Bye,
Tassilo

(*) Maybe I forgot the re-evaluation?  Well, I'll see if it occurs
again...

Re: How to debug modification to a variable value?

[Tassilo Horn]

"Davis Herring" <herring@lanl.gov> writes:

Hi Davis,

>> Now, my problem was that under some circumstances after saving, the
>> buffer-local value of `tg-schema-alist' was gone, i.e. set to nil.
>
> There's a difference between "gone" and "set to nil", even for
> automatically-buffer-local variables.  Does it still have a
> buffer-local value (which is wrong), or is it using the default
> (again)?  C-h v will say.

"Gone" means, it has the buffer-local value nil.  So there seems to be
no call to `kill-local-variable'.

>> I double-checked `greql-set-fontlock-types-regex' that it doesn't
>> modify `tg-schema-alist', and it doesn't.  I also removed all
>> destructive function calls in there, although it operates only on a
>> list created by `mapcar', and that's a copy anyway, right?
>
> Destructive operations on lists can't set the value of a variable to nil,
> nor can they invoke `kill-local-variable'.  `set[qf]?' is of course
> destructive, as are `kill-local-variable' and `kill-all-local-variables';
> have you looked for all of those?

There's no `kill-[all-]local-variable[s]', and the `setq' of that
variable is isolated in one command (see end of this message), which is
never called internally, only by a keybinding.

>> So what I need is some way to be put in the debugger when the value
>> of `tg-schema-alist' is modified.  Is that feasible?  I tried adding
>> an after advice to `setq' which does exactly that, but that screwed
>> my emacs instance.  I guess it's no good idea to advice such
>> primitives...
>
> I don't that you can do this in the Emacs debugger, but you should be
> able to use a watchpoint in gdb for this purpose.  Find where the
> buffer-local value of the variable is stored, set the watchpoint, and
> then save.

Sounds good, but how do I do that?  Especially the "find where the
buffer-local value of the variable is stored" part...

> Meanwhile, my psychic powers suggest that you have a `let' binding of
> the variable in question and switch buffers within the `let', or else
> that something is reasserting your major mode so that
> `kill-all-local-variables' is getting called.

Hm, the problem occured while I was using many different frames, which I
didn't use till now, so something in this direction seems possible.  But
between the variable is set buffer-locally to some alist and the
variable is set buffer-locally to nil, there was only one C-x C-s
without switching frames/buffers at all.

> You could test for the latter with the `permanent-local' property.

Oh, I didn't know that property.

,----[ (info "(elisp)Creating Buffer-Local") ]
|    A buffer-local variable is "permanent" if the variable name (a
| symbol) has a `permanent-local' property that is non-`nil'.  Permanent
| locals are appropriate for data pertaining to where the file came from
| or how to save it, rather than with how to edit the contents.
`----

I don't know if this is appropriate for `tg-schema-alist'.  It
determines the completion possibilities and highlighting, and is not
strictly related to the current buffer/file, but to another file a user
can select and switch.

> PS - Sorry if some of these instructions are obvious;

They are not, at least to me.  So thanks a lot!

> I don't know just what you know!

Sure, I just wantet to avoid posting the whole code, if not absolutely
neccessary.  It's free sofware, and you can svn checkout it from [1]
using anonymous/secret as user/password.

But basically, there are two modes: tg-mode has a simple parser for a
schema description language, and greql-mode which is a mode for a graph
query language, which is used for querying graphs conforming to some TG
schema.

So tg-mode contains

--8<---------------cut here---------------start------------->8---
(defvar tg-schema-alist nil
  "The schema of the current TG file.")
(make-variable-buffer-local 'tg-schema-alist)
--8<---------------cut here---------------end--------------->8---

and in greql-mode (which requires tg-mode), there is the only function
modifying the value of `tg-schema-alist' by assigning the buffer-local
value of a temporary buffer to the buffer-local value in the query
buffer.

--8<---------------cut here---------------start------------->8---
(defun greql-set-graph (graph)
  "Set `greql-graph' to GRAPH and parse it with `tg-parse-schema'."
  (interactive "fGraph file: ")
  (setq greql-graph graph)
  (let ((g greql-graph)
        schema-alist unique-name-map)
    (with-temp-buffer
      (insert-file-contents g)
      (tg-init-schema)
      (setq schema-alist tg-schema-alist)
      (setq unique-name-map tg-unique-name-hashmap))
    (setq tg-schema-alist schema-alist)
    (setq tg-unique-name-hashmap unique-name-map))
  ;; Setup schema element font locking
  (greql-set-fontlock-types-regex))
--8<---------------cut here---------------end--------------->8---

Bye,
Tassilo
__________
[1] https://svn.uni-koblenz.de/ist/projects/jgralab/trunk/utils

Re: How to debug modification to a variable value?

[Stefan Monnier]

> The main question still stands: is it possible to debug the modification
> of a variable's value?  That would also find the place in the code where
> a destructive function causes harm.

No, there is currently no such hook.
It wouldn't be terribly hard to add one, I think.
Patches welcome,


        Stefan

Re: How to debug modification to a variable value?

[alin.s]

I suggest you to use the WATCH in gdb for modifications of the variable,
AWATCH for reading of the variable, etc.

To detect the location of the variable is easy: install a breakpoint in
make-variable-buffer-local that stops when exactly the variable you are
interested about is set, then see there the location of tg-schema-alist.
Afterward you can use the x* functions from .gdbinit of emacs to debug your
problem...






Tassilo Horn-5 wrote:
> 
> Hi all,
> 
> today I had some strange problems with a major mode we use at my
> institute, and that I created and maintain.  This mode defines some
> buffer-local variable:
> 
> --8<---------------cut here---------------start------------->8---
> (defvar tg-schema-alist nil
>   "The schema of the current TG file.")
> (make-variable-buffer-local 'tg-schema-alist)
> --8<---------------cut here---------------end--------------->8---
> 
> This variable is set once, when a new graph is loaded (whatever that
> means, but there is only one function that setqs the variable).
> 
> Also, there's a function, which refreshes some font-locking stuff when
> saving.  This function is added to `after-save-hook' locally.
> 
> --8<---------------cut here---------------start------------->8---
> after-save-hook is a variable defined in `files.el'.
> Its value is 
> (t greql-set-fontlock-types-regex)
> 
> Local in buffer method-calls-per-activity.greql; global value is nil
> --8<---------------cut here---------------end--------------->8---
> 
> Now, my problem was that under some circumstances after saving, the
> buffer-local value of `tg-schema-alist' was gone, i.e. set to nil.
> 
> I double-checked `greql-set-fontlock-types-regex' that it doesn't modify
> `tg-schema-alist', and it doesn't.  I also removed all destructive
> function calls in there, although it operates only on a list created by
> `mapcar', and that's a copy anyway, right?
> 
> Till now, I couldn't reproduce this behavior.  After restarting emacs,
> the problem was gone first.  Suddenly, it appeared again.  Maybe it has
> something to do with using the same buffer in many frames (tty and X),
> but I cannot tell for sure...
> 
> So what I need is some way to be put in the debugger when the value of
> `tg-schema-alist' is modified.  Is that feasible?  I tried adding an
> after advice to `setq' which does exactly that, but that screwed my
> emacs instance.  I guess it's no good idea to advice such primitives...
> 
> Bye,
> Tassilo
> 
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/How-to-debug-modification-to-a-variable-value--tp27313386p27324834.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: How to debug modification to a variable value?

[Richard Stallman]

    It wouldn't be terribly hard to add one, I think.

It would be very bad to use this hook for anything bug debugging,
since it means that setting a variable could do anything whatsoever.
So if such a featur one is added, it will be very important to make
sure it isn't used except for debugging.

One way is to design the feature so it always calls the debugger
when activated.  For instance, if a certain flag is set in a symbol,
then setting its value or function slot calls the debugger.

Re: How to debug modification to a variable value?

[alin.s]

Stefan Monnier wrote:
> 
> No, there is currently no such hook.
> It wouldn't be terribly hard to add one, I think.
> Patches welcome,
> 

I can do it, and if nobody else starts it, I will make it when I have time.


-- 
View this message in context: http://old.nabble.com/How-to-debug-modification-to-a-variable-value--tp27313386p27327035.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: How to debug modification to a variable value?

[Tassilo Horn]

"alin.s" <alinsoar@voila.fr> writes:

> I suggest you to use the WATCH in gdb for modifications of the
> variable, AWATCH for reading of the variable, etc.
>
> To detect the location of the variable is easy: install a breakpoint
> in make-variable-buffer-local that stops when exactly the variable you
> are interested about is set, then see there the location of
> tg-schema-alist.  Afterward you can use the x* functions from .gdbinit
> of emacs to debug your problem...

Thanks for that explanation.  I'll try that out as soon as I find some
time.  And also thanks for volunteering on implementing a debug facility
for cases like that.

Bye,
Tassilo

Re: How to debug modification to a variable value?

[Stefan Monnier]

>     It wouldn't be terribly hard to add one, I think.
> It would be very bad to use this hook for anything bug debugging,
> since it means that setting a variable could do anything whatsoever.
> So if such a featur one is added, it will be very important to make
> sure it isn't used except for debugging.

> One way is to design the feature so it always calls the debugger
> when activated.  For instance, if a certain flag is set in a symbol,
> then setting its value or function slot calls the debugger.

I was thinking of a variant of read-only, so `setq' (and `let') would
signal an error (and hence throw you in the debugger).


        Stefan

Re: How to debug modification to a variable value?

[Davis Herring]

> "Gone" means, it has the buffer-local value nil.  So there seems to be
> no call to `kill-local-variable'.

That narrows the possibilities, although `make-local-variable' could have
happened later.

> There's no `kill-[all-]local-variable[s]', and the `setq' of that
> variable is isolated in one command (see end of this message), which is
> never called internally, only by a keybinding.

Don't forget `set'.  It's unlikely but not impossible some generic code is
doing something like (set (caar (buffer-local-variables)) nil).  So of
course check with "emacs -Q", and enable as little as possible that might
be trying to be clever like that.

> Sounds good, but how do I do that?  Especially the "find where the
> buffer-local value of the variable is stored" part...

(I think this has been adequately addressed by others.)

>> Meanwhile, my psychic powers suggest that you have a `let' binding of
>> the variable in question and switch buffers within the `let', or else
>> that something is reasserting your major mode so that
>> `kill-all-local-variables' is getting called.
>
> Hm, the problem occured while I was using many different frames, which I
> didn't use till now, so something in this direction seems possible.  But
> between the variable is set buffer-locally to some alist and the
> variable is set buffer-locally to nil, there was only one C-x C-s
> without switching frames/buffers at all.

And you're sure that there are no relevant `let's?  There's actually a bug
there, which I failed to actually mention: see
http://debbugs.gnu.org/cgi-bin/bugreport.cgi?bug=3467

>> You could test for the latter with the `permanent-local' property.
>
> Oh, I didn't know that property.
> [...]
> I don't know if this is appropriate for `tg-schema-alist'.  It
> determines the completion possibilities and highlighting, and is not
> strictly related to the current buffer/file, but to another file a user
> can select and switch.

I didn't mean that your variables should really use it.  Just that you
could put it on them for debugging to see whether they were getting killed
by `kill-all-local-variables', since it will spare permanent locals.  Of
course, you say it still has a local value, so that's probably not
happening, but you could check anyway, for the odd case of
`kill-all-local-variables' + `make-local-variable' (not
`make-variable-buffer-local') that looks very much like `setq' with nil.

> So tg-mode contains
> [...]
> and in greql-mode (which requires tg-mode), there is the only function
> modifying the value of `tg-schema-alist' by assigning the buffer-local
> value of a temporary buffer to the buffer-local value in the query
> buffer.

Aha!  Now we can ignore most of what I've written so far (except perhaps
for reference for future bugs).

`tg-init-schema' also sets `tg-schema-alist', of course; that's how
`greql-set-graph' gets it.  And I see that `tg-eldoc-init' adds that
function to `after-save-hook' (globally!), and that `tg-initialize' adds
-that- to `eldoc-mode-hook' (globally!), and that `tg-mode' calls -that-.

While greql-mode.el never actually invokes `tg-mode' for you, I suspect
that at some point your Emacs did run `tg-mode', which called
`tg-initialize', which registered `tg-eldoc-init', which got called
(perhaps in some other buffer) by `eldoc-mode' and registered
`tg-init-schema', which got called by `basic-save-buffer' in your GReQL
buffer.  This non-local, delayed effect probably does look a lot like
magic and probably does only happen in some Emacs sessions.

Am I right?

Davis

PS - (lambda (elem) (car elem)) is usually just written 'car or #'car.

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: How to debug modification to a variable value?

[Andreas Roehler]

[ ... ]
> The main question still stands: is it possible to debug the modification
> of a variable's value?  That would also find the place in the code where
> a destructive function causes harm.
> 
> Bye,
> Tassilo
> 
> (*) Maybe I forgot the re-evaluation?  Well, I'll see if it occurs
> again...
> 
> 
> 

Hi Tassilo,

what about to run a form with "assert MY-Values last-command this-command"
with a timer in background, so it's signaled as soon the var is changed?

Andreas

Re: How to debug modification to a variable value?

[Tassilo Horn]

"Davis Herring" <herring@lanl.gov> writes:

Hi Davis,

>> So tg-mode contains
>> [...]
>> and in greql-mode (which requires tg-mode), there is the only function
>> modifying the value of `tg-schema-alist' by assigning the buffer-local
>> value of a temporary buffer to the buffer-local value in the query
>> buffer.
>
> Aha!  Now we can ignore most of what I've written so far (except perhaps
> for reference for future bugs).
>
> `tg-init-schema' also sets `tg-schema-alist', of course; that's how
> `greql-set-graph' gets it.  And I see that `tg-eldoc-init' adds that
> function to `after-save-hook' (globally!),

Indeed, that's wrong.  Now it's only added to `after-save-hook' locally.

> and that `tg-initialize' adds -that- to `eldoc-mode-hook' (globally!),
> and that `tg-mode' calls -that-.

No, `tg-eldoc-init' is added locally, but I guess you mean that as a
result, `tg-init-schema' ends up globally in `after-save-hook' as soon
as `eldoc-mode-hook' was run, and you are right.

> While greql-mode.el never actually invokes `tg-mode' for you, I suspect
> that at some point your Emacs did run `tg-mode', which called
> `tg-initialize', which registered `tg-eldoc-init', which got called
> (perhaps in some other buffer) by `eldoc-mode' and registered
> `tg-init-schema', which got called by `basic-save-buffer' in your GReQL
> buffer.  This non-local, delayed effect probably does look a lot like
> magic and probably does only happen in some Emacs sessions.
>
> Am I right?

Absolutely.  How much time did you spend on that, and how did you debug
it?  Using a watchpoint in gdb as you suggested?  While I clearly see
that the global `add-hook' is plain wrong, I wouldn't have found it by
looking carefully at the code.

Thanks a lot, I evaled

  (cons "Davis Herring" list-of-extremely-helpful-guys).

Bye,
Tassilo

Re: How to debug modification to a variable value?

[alin.s]

Tassilo Horn-5 wrote:
> 
>> To detect the location of the variable is easy: install a breakpoint
>> in make-variable-buffer-local that stops when exactly the variable you
>> are interested about is set, then see there the location of
>> tg-schema-alist.  Afterward you can use the x* functions from .gdbinit
>> of emacs to debug your problem...
> 
> Thanks for that explanation.  I'll try that out as soon as I find some
> time.  And also thanks for volunteering on implementing a debug facility
> for cases like that.
> 

I will not start new problems soon, because I am busy with my current
problems. 




-- 
View this message in context: http://old.nabble.com/How-to-debug-modification-to-a-variable-value--tp27313386p27335402.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: How to debug modification to a variable value?

[Tassilo Horn]

Andreas Roehler <andreas.roehler@online.de> writes:

Hi Andreas,

> what about to run a form with "assert MY-Values last-command
> this-command" with a timer in background, so it's signaled as soon the
> var is changed?

I think that timer would need to run every fraction of a second to have
a chance to point me in the right direction.

Have a look at Davis last message.  He spotted the error: a small
function was added to `after-save-hook' globally, but it must be there
only buffer-locally.  The assert has to be evaluated exactly when that
function is executed, to point me into the right direction.

I guess the watchpoint approach is more exact, and I'll learn how to do
that as soon as I find some time.

Bye,
Tassilo

Re: How to debug modification to a variable value?

[Andreas Roehler]

Tassilo Horn wrote:
> Andreas Roehler <andreas.roehler@online.de> writes:
> 
> Hi Andreas,
> 
>> what about to run a form with "assert MY-Values last-command
>> this-command" with a timer in background, so it's signaled as soon the
>> var is changed?
> 
> I think that timer would need to run every fraction of a second to have
> a chance to point me in the right direction.
> 
> Have a look at Davis last message.  He spotted the error: a small
> function was added to `after-save-hook' globally, but it must be there
> only buffer-locally.  The assert has to be evaluated exactly when that
> function is executed, to point me into the right direction.
> 
> I guess the watchpoint approach is more exact, 

[ ... ]

Probably,

anyway, as my small test case works so far, here the example tracking
the value of var "count":

(BTW form with "assert" don't work, why?)

(setq count '1)

(defvar zeit nil)

(defun ausgabe ()
  ;; (assert (eq count 1))
  (message "%s" "increase count, assert should fail")
  (setq count (1+ count))
  (message "%s %s %s %d" this-command last-command (eq count 1) count))

(if zeit
    (progn (cancel-timer zeit)  (setq zeit nil))
  (setq zeit (run-with-timer 0.5 1 'ausgabe)))




Andreas

Re: How to debug modification to a variable value?

[Davis Herring]

>> and that `tg-initialize' adds -that- to `eldoc-mode-hook' (globally!),
>> and that `tg-mode' calls -that-.
>
> No, `tg-eldoc-init' is added locally, but I guess you mean that as a
> result, `tg-init-schema' ends up globally in `after-save-hook' as soon
> as `eldoc-mode-hook' was run, and you are right.

Actually, I just didn't notice that one of the `add-hook's was local;
since I had noticed the other one was global, I didn't double-check this
one.

> Absolutely.  How much time did you spend on that, and how did you debug
> it?  Using a watchpoint in gdb as you suggested?  While I clearly see
> that the global `add-hook' is plain wrong, I wouldn't have found it by
> looking carefully at the code.

Maybe 15 minutes?  I wasn't counting.  I just thought it was odd that the
"only" function that set `tg-schema-alist' wasn't even in the same file
where it was defvar-ed, so I looked in tg-mode.el, found `tg-init-schema'
(which I could really have guessed from `greql-set-graph'), and then
looked for references to that function.  As soon as I saw it put on
`after-save-hook', it was obvious (though I had to look at the other hook
to understand the whole sequence).  I never left the browser window
wherein I browsed your SVN repository except to check Emacs'
documentation.

> Thanks a lot, I evaled
>
>   (cons "Davis Herring" list-of-extremely-helpful-guys).

Am I to be garbage collected, then?  You need a `setq' (like `push' does);
perhaps I can still be rescued from `values'.  ;)

Glad I could help,
Davis

PS - Your messages to me (and the list) bear the header
Mail-Followup-To: herring@lanl.gov, emacs-devel@gnu.org
...which seems wrong (especially when I am the one replying!); surely
Mail-Followup-To: tassilo@member.fsf.org, emacs-devel@gnu.org
is what you want?

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.