blink-cursor-end sometimes fails and disrupts pre-command-hook

blink-cursor-end sometimes fails and disrupts pre-command-hook

[Ken Manheimer]

i've been having a hell of a time tracking down a
user-configuration-sensitive blink-cursor error, and can't even report
a case to reproduce it.  i do have a simple change which prevents the
problem and which i think is proper, though i couldn't swear to that.

i'm attaching the patch below.

after my first keystroke in an emacs session, i find the error message
in the *Messages* buffer:

Error in pre-command-hook: (error Invalid timer)

deactivation of the pre-command-hook disrupts my editor session, of
course, disabling all activity on the pre-command-hook.

after a bunch of searching, i determined the problem was happening in
blink-cursor-end running off of the pre-command-hook.  i was unable to
determine the combination of packages and settings from my emacs rc
file necessary to trigger it (and i spent a few hours adding things in
and out, aaargh).

it started happening about a week ago, with upgrade to a recent
version of emacs 22.

one incidental point.

i would really, really, *really* have liked to backtraces to help
track this down, but emacs backtraces are useless for that by design.
you have to know where to put them to get the backtrace from an error
at that point.  that is terrible - more than half the usefulness of a
backtrace is in pinpointing where an error occurred.  i can understand
not wanting to produce a backtrace on every error, if it's an
expensive operation.  however, i can't understand not having a mode -
eg, with a `debug-on-error' style variable, eg
`retain-backtrace-on-error' - for retaining backtraces on any error,
to be replaced when the next error happens.   i have no C chops, so
can't implement this, but would love it if someone were interested in
doing so.  (if this kind of thing is already possible, please someone
clue me in, and my apologies for my rant.)

here's the change which prevents the blink-cursor problem.  the
condition is similar to that used in blink-cursor-mode, and i'm pretty
sure is not harmful.  it may be covering over some kind of timing
sensitivity in changes to pre-command-hook, but i've been unable to
track that down.
-- 
ken
ken.manheimer@gmail.com
http://myriadicity.net

--- frame.el	21 Jul 2006 03:42:52 -0400	1.237
+++ frame.el	18 Aug 2006 17:58:18 -0400	
@@ -1308,7 +1308,8 @@
 itself as a pre-command hook."
   (remove-hook 'pre-command-hook 'blink-cursor-end)
   (internal-show-cursor nil t)
-  (cancel-timer blink-cursor-timer)
+  (if blink-cursor-timer
+      (cancel-timer blink-cursor-timer))
   (setq blink-cursor-timer nil))

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Chong Yidong]

"Ken Manheimer" <ken.manheimer@gmail.com> writes:

> i've been having a hell of a time tracking down a
> user-configuration-sensitive blink-cursor error, and can't even report
> a case to reproduce it.
>
> after my first keystroke in an emacs session, i find the error message
> in the *Messages* buffer:
>
> Error in pre-command-hook: (error Invalid timer)
>
> after a bunch of searching, i determined the problem was happening in
> blink-cursor-end running off of the pre-command-hook.

If blink-cursor-end is being run, that means that blink-cursor-start
has been called.  If blink-cursor-start completed, blink-cursor-timer
should be non-nil.  There isn't a lot of possibilities for
blink-cursor-timer to be nil inside blink-cursor-end.

One possibility is a rogue interrupt handler:

  * blink-cursor-end is interrupted before it can cancel
    blink-cursor-timer; or blink-cursor-start is interrupted before it
    can set blink-cursor-timer
  * Something inside the interrupt handler runs pre-command-hook, so
    blink-cursor-end runs, and blink-cursor-timer is set to nil.
  * Execution resumes.  blink-cursor-timer is now nil.

However, I don't know any way an interrupt handler could end up
running a pre-command-hook.

So this is very strange.

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Chong Yidong]

"Ken Manheimer" <ken.manheimer@gmail.com> writes:

> i see now - blink-cursor-start is called by the timer, which sets
> blink-cursor-end on the pre-command-hook, does some cursor visibility
> business, and then reestablishes the blink-cursor-timer.
>
> seems to me it would be more reliable to set the blink-cursor-timer
> before adding blink-cursor-end to the pre-command-hook!
>
> i just tried making this change and removing the change from
> blink-cursor-end (by putting revised copies of both functions in my
> emacs rc), and the problem is not happening.  i would suggest going
> with both changes - blink-cursor-end with the condition i suggested
> adding, and blink-cursor-start looking like:

Bingo --- I think that's it.  Something in your customizations must be
causing the run-with-timer call in blink-cursor-start to signal an
error.  One possibility is an invalid `blink-cursor-interval'
variable.  Since blink-cursor-start is run from an (idle) timer,
quitting is disabled.

The question is, why did this surface only with the latest Emacs
update?

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Ken Manheimer]

On 8/19/06, Chong Yidong <cyd@stupidchicken.com> wrote:
> "Ken Manheimer" <ken.manheimer@gmail.com> writes:
>
> > i see now - blink-cursor-start is called by the timer, which sets
> > blink-cursor-end on the pre-command-hook, does some cursor visibility
> > business, and then reestablishes the blink-cursor-timer.
> >
> > seems to me it would be more reliable to set the blink-cursor-timer
> > before adding blink-cursor-end to the pre-command-hook!
> >
> > i just tried making this change and removing the change from
> > blink-cursor-end (by putting revised copies of both functions in my
> > emacs rc), and the problem is not happening.  i would suggest going
> > with both changes - blink-cursor-end with the condition i suggested
> > adding, and blink-cursor-start looking like:
>
> Bingo --- I think that's it.  Something in your customizations must be
> causing the run-with-timer call in blink-cursor-start to signal an
> error.  One possibility is an invalid `blink-cursor-interval'
> variable.  Since blink-cursor-start is run from an (idle) timer,
> quitting is disabled.
>
> The question is, why did this surface only with the latest Emacs
> update?

actually, it isn't necessarily the latest emacs update, though it
probably is a one within the last several weeks.  i started noticing
the problem a week or so ago, but didn't have time to investigate
until yesterday.  i may not have done an update for a week or more
until before i first started seeing the problem.
-- 
ken
ken.manheimer@gmail.com
http://myriadicity.net

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Chong Yidong]

>> The question is, why did this surface only with the latest Emacs
>> update?
>
> actually, it isn't necessarily the latest emacs update, though it
> probably is a one within the last several weeks.  i started noticing
> the problem a week or so ago, but didn't have time to investigate
> until yesterday.  i may not have done an update for a week or more
> until before i first started seeing the problem.

Could you try to track down why the error happens?

One way I suggest doing this is to go through the various `error'
calls inside run-with-timer, and make them additionally print a
message or save something to a variable.  If we can find out which
`error' is being triggered, we can figure out whether it's a problem
with Emacs, or with your customizations.

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Ken Manheimer]

On 8/19/06, Chong Yidong <cyd@stupidchicken.com> wrote:
> >> The question is, why did this surface only with the latest Emacs
> >> update?
> >
> > actually, it isn't necessarily the latest emacs update, though it
> > probably is a one within the last several weeks.  i started noticing
> > the problem a week or so ago, but didn't have time to investigate
> > until yesterday.  i may not have done an update for a week or more
> > until before i first started seeing the problem.
>
> Could you try to track down why the error happens?
>
> One way I suggest doing this is to go through the various `error'
> calls inside run-with-timer, and make them additionally print a
> message or save something to a variable.  If we can find out which
> `error' is being triggered, we can figure out whether it's a problem
> with Emacs, or with your customizations.

i'll do that, soon (i've run out of time, today).

(incidentally, did my rant about `backtrace' make sense to anyone, and
if so, might it be feasible to fix it?  my model is python's exception
mechanism, which enables you to obtain the backtrace of an error at
the point that the exception is caught, which is an extremely valuable
introspection tool for debugging and executive operation...)

-- 
ken
ken.manheimer@gmail.com
http://myriadicity.net

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Richard Stallman]

    i would really, really, *really* have liked to backtraces to help
    track this down, but emacs backtraces are useless for that by design.
    you have to know where to put them to get the backtrace from an error
    at that point.

I do not understand that sentence.  What do you mean, "put them"?
What do you mean by "where"--what is a "place"?

      however, i can't understand not having a mode -
    eg, with a `debug-on-error' style variable, eg
    `retain-backtrace-on-error' - for retaining backtraces on any error,
    to be replaced when the next error happens.

We could do that easily, I suppose, but could you explain
how it relates to the problem?

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Ken Manheimer]

On 8/20/06, Richard Stallman <rms@gnu.org> wrote:
>     i would really, really, *really* have liked to backtraces to help
>     track this down, but emacs backtraces are useless for that by design.
>     you have to know where to put them to get the backtrace from an error
>     at that point.
>
> I do not understand that sentence.  What do you mean, "put them"?
> What do you mean by "where"--what is a "place"?

"put them" == put the `(backtrace)' call
"where" == at the point in the code for which you want the backtrace.

what i want, instead, is the ability to get a stack trace for an error
when i am handling that error, in a condition-case or in the
interactive interpreter.  i want to be able to examine the backtrace
to help find where/when the error happened, rather than needing to
first know where it's happening and then situate a backtrace call, for
the next time.

in fact, this seems like such a key use of backtraces that i suspect
i'm misreading the manual, and that there is a way to do this that i'm
missing.

>       however, i can't understand not having a mode -
>     eg, with a `debug-on-error' style variable, eg
>     `retain-backtrace-on-error' - for retaining backtraces on any error,
>     to be replaced when the next error happens.
>
> We could do that easily, I suppose, but could you explain
> how it relates to the problem?

it would have been helpful in identifying where the "error in
pre-command-hook: (error invalid timer)" was happening.

as it is, i floundered a lot tracking it down, having to first
identify that the error was happening in the blink-paren-end - if i
could have asked emacs to keep the last stack trace in a variable, it
would have been trivial to idenitify that.  further, i had to remove
blink-paren-end from the pre-command-hook via my emacs rc so i could
run it "manually", with the debugger going, so i could examine the
error condition.

even now, as chong yidong suggests, i need to identify where use of a
timer is going awry, disrupting the blink-cursor dance, so i have to
instrument any possibly related error call that might be happening to
see which one it is, and then examine that context.  (the calls may be
in preloaded lisp, which further impedes the investigation.)  if i
could have emacs give me the stack trace after any error, that would
enable me to unravel these layers of information without digging
through and changing code.

this is not my own idea - python does something very like what i'm
describing.  see http://docs.python.org/lib/module-traceback.html for
the basics.

-- 
ken
ken.manheimer@gmail.com
http://myriadicity.net

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[David Kastrup]

"Ken Manheimer" <ken.manheimer@gmail.com> writes:

> On 8/20/06, Richard Stallman <rms@gnu.org> wrote:
>>     i would really, really, *really* have liked to backtraces to help
>>     track this down, but emacs backtraces are useless for that by design.
>>     you have to know where to put them to get the backtrace from an error
>>     at that point.
>>
>> I do not understand that sentence.  What do you mean, "put them"?
>> What do you mean by "where"--what is a "place"?
>
> "put them" == put the `(backtrace)' call
> "where" == at the point in the code for which you want the backtrace.
>
> what i want, instead, is the ability to get a stack trace for an error
> when i am handling that error, in a condition-case or in the
> interactive interpreter.

(setq debug-on-signal t)

It's not actually a fun setting, since it catches quite a lot, but it
is useful.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Richard Stallman]

    "put them" == put the `(backtrace)' call
    "where" == at the point in the code for which you want the backtrace.

To put in an explicit call to (backtrace) is an unusual technique, but
I don't see why it would not work.  Since it outputs to
standard-output, you could save multiple backtraces in one buffer.

    what i want, instead, is the ability to get a stack trace for an error
    when i am handling that error, in a condition-case or in the
    interactive interpreter.

Does (setq debug-on-signal t) do what you want?
If not, precisely how does what you want differ from that?

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Ken Manheimer]

On 8/21/06, Richard Stallman <rms@gnu.org> wrote:
>     "put them" == put the `(backtrace)' call
>     "where" == at the point in the code for which you want the backtrace.
>
> To put in an explicit call to (backtrace) is an unusual technique, but
> I don't see why it would not work.  Since it outputs to
> standard-output, you could save multiple backtraces in one buffer.
>
>     what i want, instead, is the ability to get a stack trace for an error
>     when i am handling that error, in a condition-case or in the
>     interactive interpreter.
>
> Does (setq debug-on-signal t) do what you want?
> If not, precisely how does what you want differ from that?

it requires user intervention, rather than enbling me to handle the
backtrace in the condition-case, or stash the backtrace for later
examination by whatever/whoever.

this is not a gratuitous concern.  i have a condition-case in the
post-command hook by which the allout extensions i'm developing
cooperate with allout-mode.  the condition-case catches any errors, so
that it and other stuff on the post-command-hook are not disrupted by
errors in my extensions' post-command business.  for now i stash the
error information in a variable and post a message about the
occurrence, but i would much prefer to stash a complete traceback
implicating the error caught by the condition-case.

that said, i had failed to find/realize that debug-on-signal would
cause a debug session at the point of error within the condition case.
 while not quite what i'm seeking, that will be helpful.  is there a
reasonable way to substitute a function for the debug session, so it
could stash a backtrace and proceed on, rather than invoking user
intervention?

-- 
ken
ken.manheimer@gmail.com
http://myriadicity.net

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Stuart D. Herring]

> (incidentally, did my rant about `backtrace' make sense to anyone, and
> if so, might it be feasible to fix it?  my model is python's exception
> mechanism, which enables you to obtain the backtrace of an error at
> the point that the exception is caught, which is an extremely valuable
> introspection tool for debugging and executive operation...)

If `debug-on-error' is non-nil, Emacs will stop as soon as an error is
signaled, unless a `condition-case' catches it.  Otherwise you get the
complete backtrace.  What more than that do you want?  Are you asking for
some programmatic interface to what would be displayed if you didn't catch
an error?  If it's okay to look at things before an error is signaled, you
might be interested in `backtrace-frame'.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Stuart D. Herring]

> that said, i had failed to find/realize that debug-on-signal would
> cause a debug session at the point of error within the condition case.
>  while not quite what i'm seeking, that will be helpful.  is there a
> reasonable way to substitute a function for the debug session, so it
> could stash a backtrace and proceed on, rather than invoking user
> intervention?

C-h v debugger

Does that help?

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Ken Manheimer]

On 8/21/06, Stuart D. Herring <herring@lanl.gov> wrote:
> > that said, i had failed to find/realize that debug-on-signal would
> > cause a debug session at the point of error within the condition case.
> >  while not quite what i'm seeking, that will be helpful.  is there a
> > reasonable way to substitute a function for the debug session, so it
> > could stash a backtrace and proceed on, rather than invoking user
> > intervention?
>
> C-h v debugger
>
> Does that help?

that was the final piece i needed.

i've just barely started using it, but my post-command-hook has this fragment:

   (condition-case failure
         [...]
          (when allout-widgets-changes-record
            (let* ((debug-on-signal t)
                   (debug-on-error t)
                   (debugger 'allout-widgets-postcommand-hook-error-handler)
              [process the pending changes]
           [...]
            ))
     (error (setq allout-widgets-changes-record nil))

and the handler looks like this:

(defun allout-widgets-postcommand-hook-error-handler (mode args)
  "Process errors which occurred within the extent of a command hook.

We store a backtrace of the error information in the variable,
`allout-widgets-last-hook-error', unset the error handlers, and
reraise the error, so that processing continues to the
encompassing condition-case."
  ;; first deconstruct special error environment so errors here propagate
  ;; to encompassing condition-case:
  (setq debugger 'debug
        debug-on-error nil
        debug-on-signal nil)
  (let* ((bt (with-output-to-string (backtrace)))
         (this "allout-widgets-postcommand-hook-error-handler")
         (header
          (format "allout-widgets-last-hook-error stored and posted, %s/%s %s"
                  this mode args
                  (format-time-string "%e-%b-%Y %r" (current-time)))))
    ;; post to *Messages* then immediately replace with more compact notice:
    (message (setq allout-widgets-last-hook-error
                   (format "%s:\n%s" header bt)))
    (message header) (sit-for allout-widgets-hook-error-post-time)
    ;; reraise the error, or one concerning this function if unexpected:
    (if (equal mode 'error)
        (apply 'signal args)
      (error "%s: unexpected mode, %s %s" this mode args))))

i expect this to be very helpful tracking down errors while making my
post-command-hook a "good citizen", unlikely to disrupt the hook due
to uncaught errors.

thanks all for pointing out what i needed to do to record backtraces
at the point of failure.
-- 
ken
ken.manheimer@gmail.com
http://myriadicity.net

Re: blink-cursor-end sometimes fails and disrupts pre-command-hook

[Richard Stallman]

I think it would be a useful thing to set up a facility to save
backtraces in (more or less) this way, one that users could easily enable
for certain filters, for certain timers, and for certain hooks.

I will put that in etc/TODO, and maybe someone would like to start
working on it for installation after Emacs 22 is released.