watching for variable assignment

watching for variable assignment

[Eric Abrahamsen]

I've been fighting gnus recently over mail splitting, trying to keep the
variable nnimap-split-fancy set to my preferred value -- as I use gnus,
some other part of the code keeps setting it to "nil" and messing up my
splits.

I'd really like to know what's doing that, so I'm trying to somehow
trigger a message or a backtrace whenever that variable's value changes.
I tried advising "set" (thinking that set was the most fundamental
function for variable assignment), but that doesn't seem to get called.
I advised "setq", but that gave me a recursion exception: defadvice
probably uses setq itself.

What are my other options? How do I figure out who keeps blanking this
variable out?

Thanks,
Eric

Using GNU Emacs 24.3.1. Hmm, maybe if I tried emacs from git, and the
new add-function stuff...

Re: watching for variable assignment

[Jambunathan K]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I'm trying to somehow trigger a message or a backtrace whenever that
> variable's value changes.


You are looking for a "Write Breakpoint".  I haven't done much Elisp
debugging.  Going by the docs, the following looks promising.

    (info "(elisp) Global Break Condition")

Re: watching for variable assignment

[Eric Abrahamsen]

Jambunathan K <kjambunathan@gmail.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I'm trying to somehow trigger a message or a backtrace whenever that
>> variable's value changes.
>
>
> You are looking for a "Write Breakpoint".  I haven't done much Elisp
> debugging.  Going by the docs, the following looks promising.
>
>     (info "(elisp) Global Break Condition")

Hey thanks! That's a new one for me, and does indeed look promising.
I've set edebug-global-break-condition to (eq nnimap-split-fancy nil),
and will see what happens. It says it tests the expression at every
"stop point". I haven't explicitly set any stop points; hopefully
they are set automatically, since the whole point of this is that I
don't know where to check for the assignment.

Re: watching for variable assignment

[Nicolas Richard]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Jambunathan K <kjambunathan@gmail.com> writes:
>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> I'm trying to somehow trigger a message or a backtrace whenever that
>>> variable's value changes.
>>
>>
>> You are looking for a "Write Breakpoint".  I haven't done much Elisp
>> debugging.  Going by the docs, the following looks promising.
>>
>>     (info "(elisp) Global Break Condition")
>
> Hey thanks! That's a new one for me, and does indeed look promising.
> I've set edebug-global-break-condition to (eq nnimap-split-fancy nil),
> and will see what happens. It says it tests the expression at every
> "stop point". I haven't explicitly set any stop points; hopefully
> they are set automatically, since the whole point of this is that I
> don't know where to check for the assignment.

I guess it requires the relevant code to be instrumented for edebug,
otherwise edebug won't do anything.

-- 
Nico.

Re: watching for variable assignment

[Eric Abrahamsen]

Nicolas Richard <theonewiththeevillook@yahoo.fr> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Jambunathan K <kjambunathan@gmail.com> writes:
>>
>>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>>
>>>> I'm trying to somehow trigger a message or a backtrace whenever that
>>>> variable's value changes.
>>>
>>>
>>> You are looking for a "Write Breakpoint".  I haven't done much Elisp
>>> debugging.  Going by the docs, the following looks promising.
>>>
>>>     (info "(elisp) Global Break Condition")
>>
>> Hey thanks! That's a new one for me, and does indeed look promising.
>> I've set edebug-global-break-condition to (eq nnimap-split-fancy nil),
>> and will see what happens. It says it tests the expression at every
>> "stop point". I haven't explicitly set any stop points; hopefully
>> they are set automatically, since the whole point of this is that I
>> don't know where to check for the assignment.
>
> I guess it requires the relevant code to be instrumented for edebug,
> otherwise edebug won't do anything.

Yeah, that's what I'm realizing. A bit of a pain, but I guess I can try
to make it work... I wish there was a command that said "instrument
gnus-group-get-new-news and everything it calls", since that seems to be
where the evil happens. Anyway, I'll futz with it.

Re: watching for variable assignment

[Eric Abrahamsen]

Jambunathan K <kjambunathan@gmail.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I'm trying to somehow trigger a message or a backtrace whenever that
>> variable's value changes.
>
>
> You are looking for a "Write Breakpoint".  I haven't done much Elisp
> debugging.  Going by the docs, the following looks promising.
>
>     (info "(elisp) Global Break Condition")

No, this isn't going to work -- the only way I could find the code
that's resetting the variable would be to instrument it directly, which
would require knowing which code it was in the first place. Not much
use!

I've got git emacs installed on my system, and I think the newer version
of advising functions might work better. I'm hoping that if I add a
function to "setq" with the :before key, I can avoid the infinite loop
problem.

I read the manual, and found it pretty unhelpful on the question of how
to tell it what function you're actually trying to advise. A "setf-able
place" doesn't mean much to me when I'm just trying to hijack "setq" and
check its args.

Can someone illustrate for me the (I'd hope fairly simple) case of
adding a function :before "setq", and checking if its first argument is
"nnimap-split-fancy"? The possibility that's it's a longer setq form
with multiple arguments is one I'll consider after I've got the basic
shape down...

Thanks!
Eric

Re: watching for variable assignment

[Jambunathan K]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> nnimap-split-fancy

Why don't you rgrep for it and see where all it is getting referenced in
the source code?

Try to reduce your "sample space".  Specifically focus on parts, where
it is getting "written to" and eliminate places where it is being
"read".

Re: watching for variable assignment

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> nnimap-split-fancy
>
> Why don't you rgrep for it and see where all it is getting referenced in
> the source code?
>
> Try to reduce your "sample space".  Specifically focus on parts, where
> it is getting "written to" and eliminate places where it is being
> "read".

I just rgrep-ped.

My gut feeling is that the variable is mostly read.

Either you have to search for that variable in your custom .el files or
track the `nnmail-split-fancy' *instead*.  Summary, you are looking in
the wrong place aka re-formulate your question.

Re: watching for variable assignment

[Eric Abrahamsen]

Jambunathan K <kjambunathan@gmail.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> nnimap-split-fancy
>
> Why don't you rgrep for it and see where all it is getting referenced in
> the source code?
>
> Try to reduce your "sample space".  Specifically focus on parts, where
> it is getting "written to" and eliminate places where it is being
> "read".

Yeah, I tried that but didn't get anything conclusive. The variable is
defvar'd in gnus-sum.el, and defvoo'd in nnimap.el, and otherwise is
only let. I've put the output of a full rgrep on my .emacs.d directory
below. All the bbdb-gnus related stuff hasn't changed in literally a
decade, so I doubt it's that.

I just know that in a few days' time I'm going to be begging pardon for
wasting everyone's time on a stupid problem that was all my fault to
begin with, but... In the meantime, this is terribly frustrating!

./bbdb/lisp/bbdb-gnus.el:359:with the `nnimap-split-fancy' method you have to use macros, that is your setting
./bbdb/lisp/bbdb-gnus.el:362:\(setq nnimap-split-rule  'nnimap-split-fancy
./bbdb/lisp/bbdb-gnus.el:364:       nnimap-split-fancy
./gnus/lisp/gnus-registry.el:429:entry in `nnmail-split-fancy' or `nnimap-split-fancy', for example like
./gnus/lisp/nnimap.el:88:(defvoo nnimap-split-fancy nil
./gnus/lisp/nnimap.el:1127:			 (nnimap-split-fancy
./gnus/lisp/nnimap.el:1129:		       (nnmail-split-fancy (or nnimap-split-fancy
./gnus/lisp/nnimap.el:2028:	    (nnimap-split-fancy
./gnus/lisp/nnimap.el:2030:	  (nnmail-split-fancy (or nnimap-split-fancy
./gnus/lisp/gnus-sum.el:10699:(defvar nnimap-split-fancy)
./gnus/lisp/gnus-sum.el:10718:			 (nnimap-split-fancy
./gnus/lisp/gnus-sum.el:10720:		       (nnmail-split-fancy (or nnimap-split-fa

Re: watching for variable assignment

[Jambunathan K]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> In the meantime, this is terribly frustrating!

When going gets tough, you start asking simple questions like:

1. Did I upgrade my gnus or my own code recently?

2. Did I upgrade my emacs?

3. Do I have stale files around?  Is it loading the right library from
   the right place.

3. What happened in the last few days that made I notice this problem in
   first place.

4. Did my E-mail provider change his headers (or whatever)?

5. Did I change my PC - from Gnu/Linux to Windows to Mac etc.

Re: watching for variable assignment

[Eric Abrahamsen]

Jambunathan K <kjambunathan@gmail.com> writes:

> Jambunathan K <kjambunathan@gmail.com> writes:
>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> nnimap-split-fancy
>>
>> Why don't you rgrep for it and see where all it is getting referenced in
>> the source code?
>>
>> Try to reduce your "sample space".  Specifically focus on parts, where
>> it is getting "written to" and eliminate places where it is being
>> "read".
>
> I just rgrep-ped.
>
> My gut feeling is that the variable is mostly read.
>
> Either you have to search for that variable in your custom .el files or
> track the `nnmail-split-fancy' *instead*.  Summary, you are looking in
> the wrong place aka re-formulate your question.

Oh I definitely feel like I'm barking up the wrong tree... Hence the
sense that I'll have a "do'h!" moment before too long. But in the meantime...

Re: watching for variable assignment

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> In the meantime, this is terribly frustrating!
>
> When going gets tough, you start asking simple questions like:
>
> 1. Did I upgrade my gnus or my own code recently?

I see this post on planet.emacsen.org.

   http://ivan.kanis.fr/gnus-simple-split-unexpected-feature.html

I have feeling that you are splitting your hair over an "un-expected"
feature :-)

Re: watching for variable assignment

[Stefan Monnier]

> No, this isn't going to work -- the only way I could find the code
> that's resetting the variable would be to instrument it directly,
> which would require knowing which code it was in the first place.
> Not much use!

FWIW, I think I'd accept a patch which adds support for "watchers" on
symbols (by extending the read-only bit to 2 bits, so we can
distinguish a "really read-only symbol" from a "symbol with watchers").

> I've got git emacs installed on my system, and I think the newer version
> of advising functions might work better. I'm hoping that if I add a
> function to "setq" with the :before key, I can avoid the infinite loop
> problem.

No, you can't advise special forms (and even if you changed nadvice.el to
try and support it, it still wouldn't apply to byte-compiled code).

> I read the manual, and found it pretty unhelpful on the question of how
> to tell it what function you're actually trying to advise.  A "setf-able
> place" doesn't mean much to me when I'm just trying to hijack "setq" and
> check its args.

Can you point to the offending documentation (it's still in the process
of being written, so it would be helpful).
I don't see any "setf-able" anywhere, but FWIW such a thing is also
called "an lvalue" in other languages.

In any case it sounds like you were looking at the doc of add-function,
whereas you'd want to use advice-add instead (tho you don't really want
to use it since it wouldn't work on setq anyway).

> Can someone illustrate for me the (I'd hope fairly simple) case of
> adding a function :before "setq", and checking if its first argument is
> "nnimap-split-fancy"? The possibility that's it's a longer setq form
> with multiple arguments is one I'll consider after I've got the basic
> shape down...

It could look something like

   (defun my-setter (var val &rest _)
     (if (eq var 'nnimap-split-fancy)
         (error "Gotcha, sucker!")))
   (advice-add 'set :before #'my-setter)


-- Stefan

Re: watching for variable assignment

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> > You are looking for a "Write Breakpoint".  I haven't done much Elisp
> > debugging.  Going by the docs, the following looks promising.
> >
> >     (info "(elisp) Global Break Condition")
>
> No, this isn't going to work -- the only way I could find the code
> that's resetting the variable would be to instrument it directly, which
> would require knowing which code it was in the first place.

No, not necessarily.  You can set `edebug-all-defs' non-nil and
instrument whole libraries.  That may make things much
slower, but may be worth a try.

Michael.

Re: watching for variable assignment

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> No, this isn't going to work -- the only way I could find the code
>> that's resetting the variable would be to instrument it directly,
>> which would require knowing which code it was in the first place.
>> Not much use!
>
> FWIW, I think I'd accept a patch which adds support for "watchers" on
> symbols (by extending the read-only bit to 2 bits, so we can
> distinguish a "really read-only symbol" from a "symbol with watchers").
>
>> I've got git emacs installed on my system, and I think the newer version
>> of advising functions might work better. I'm hoping that if I add a
>> function to "setq" with the :before key, I can avoid the infinite loop
>> problem.
>
> No, you can't advise special forms (and even if you changed nadvice.el to
> try and support it, it still wouldn't apply to byte-compiled code).
>
>> I read the manual, and found it pretty unhelpful on the question of how
>> to tell it what function you're actually trying to advise.  A "setf-able
>> place" doesn't mean much to me when I'm just trying to hijack "setq" and
>> check its args.
>
> Can you point to the offending documentation (it's still in the process
> of being written, so it would be helpful).
> I don't see any "setf-able" anywhere, but FWIW such a thing is also
> called "an lvalue" in other languages.

I saw "Generalized Variable" in the first paragraph, and in my brain I
conflate "generalized variable" with "setf-able place", probably
incorrectly.

FWIW, I updated git emacs recently. The docs seem to have been greatly
expanded, and now make a lot of sense.

E

Re: watching for variable assignment

[Stefan]

> I saw "Generalized Variable" in the first paragraph, and in my brain I
> conflate "generalized variable" with "setf-able place", probably
> incorrectly.

This conflation is correct.

> FWIW, I updated git emacs recently.  The docs seem to have been greatly
> expanded, and now make a lot of sense.

Good to hear, thanks,


        Stefan