Temporarily disable `timer-event-handler'

Temporarily disable `timer-event-handler'

[Alexander Shukaev]

Hi,

I've recently discovered the dark side of Emacs: that timer events run 
by `timer-event-handler' can be triggered either at `top-level' or at 
`recursive-edit' or as part of `sit-for' (see `input-pending-p') or 
`accept-process-output' or `message' (which triggers `redisplay' and 
subsequently C function `redisplay_internal' that invokes Lisp 
interpreter machine yet again and hence potentially 
`timer-event-handler') or maybe even more.

With such plethora of possibilities for `timer-event-handler' to run 
especially in a nested manner one could even observe an interesting 
peculiarity:

- Event `A' scheduled to run ASAP.
- Event `B' scheduled to run ASAP.
- The expectation is that `B' strictly runs after `A' finishes execution.
- Well, if `A' calls `message', then according to the above, `A' could 
indirectly run `timer-event-handler' (nested) which, as a result, will 
run `B' now essentially somewhere in the middle of `A'.
- Clearly, unless taken care of, this may result in side effects and 
nasty bugs, which are difficult to track down.

Another possible side effect can be related to e.g. `let'-binding, when 
during an execution of some function some `let'-binding is performed, 
but `redisplay' is being triggered and some timer event occurs under 
that `let'-binding, which unfortunately affects it in a buggy unexpected 
way.  Also difficult to understand and hunt down.

Now, first of all, do I understand correctly that the recommended way to 
temporarily prevent timer events from happening is to `let'-bind both 
`timer-list' and `timer-idle-list' (similar to how TRAMP does it e.g. in 
`tramp-accept-process-output')?  How about a stock macro for this?

Secondly, does the above explanation, pitfall examples, and recipes to 
temporarily disable timer events in the middle of Lisp execution appear 
anywhere in the documentation?  I suspect that most users are not aware 
of this complicated design.

Finally, what's the motivation behind this design?  This looks fragile 
and error-prone to run some arbitrary code in the middle of execution of 
another code without separating their "stacks" (environment scopes), and 
even then their global side effects might interfere.  Why not only allow 
timer events to run at "safer" points of execution?  E.g. no nested 
timer events, only at `top-level', never in `redisplay', also allowed 
in-between some blessed hooks executions?

RE: Temporarily disable `timer-event-handler'

[Drew Adams]

Please don't post the same question to both help-gnu-emacs and emacs-devel.  Please pick one.  Thx.

Re: Temporarily disable `timer-event-handler'

[Stefan Monnier]

> - Event `A' scheduled to run ASAP.
> - Event `B' scheduled to run ASAP.
> - The expectation is that `B' strictly runs after `A' finishes execution.

There's the rub: currently, there's no such guarantee.  The only
"guarantee" we provide is that the processing of event A will start
before the processing of event B.

[ Note: "event" here can be a timer firing or a process output.  ]

> - Well, if `A' calls `message', then according to the above, `A' could
>   indirectly run `timer-event-handler' (nested) which, as a result, will 
>   run `B' now essentially somewhere in the middle of `A'.

If A and B are unrelated, there's no problem.
If A and B are related, there could indeed be a problem.
And notice that event B could even have been created by the processing of A.
But there can also be cases where A does want/need B to be processed
before A ends.

[ Another related situation is when A is a repeated event and a second
  A fires before the first one is done processing.  ]

> - Clearly, unless taken care of, this may result in side effects and nasty
>  bugs, which are difficult to track down.

Yup.

> Another possible side effect can be related to e.g. `let'-binding, when
>  during an execution of some function some `let'-binding is performed, 
> but `redisplay' is being triggered and some timer event occurs under that
> `let'-binding, which unfortunately affects it in a buggy unexpected way.
> Also difficult to understand and hunt down.

Oh yes, dynamically-bound vars can make that even more interesting.
Process-filters and timer events (and jit-lock, and the debugger) should
ideally be run in their own thread (and hence not affected by
dynamic-bindings in the interrupted threads).  Existing code does
occasionally depend on the current behavior, tho (e.g. in
`jit-lock-deferred-fontify` we bind `jit-lock-defer-timer` around
a call to `redisplay`), so fixing this is not completely trivial.

> Now, first of all, do I understand correctly that the recommended way to
> temporarily prevent timer events from happening is to `let'-bind both 
> `timer-list' and `timer-idle-list' (similar to how TRAMP does it e.g. in
> `tramp-accept-process-output')?  How about a stock macro for this?

The recommended way is to find a solution that doesn't involve
preventing timers from firing, so a stack macro for it doesn't sound
right (e.g. in the case of Tramp, the discussion in bug#29735 seems to
indicate that Michael agrees that disabling timers isn't "right" but
it's just what he's using so far because he wasn't able to track down
the bug in the "right" version of the code).

> I suspect that most users are not aware of this complicated design.

I don't think the design is complicated.  It's the resulting behavior
that is (arguably because the design was too naive).

> Finally, what's the motivation behind this design?  This looks fragile and
> error-prone to run some arbitrary code in the middle of execution of another
> code without separating their "stacks" (environment scopes),

It's a direct consequence of the implementation, rather than the result
of design, AFAICT.

> Why not only allow timer events to run at "safer" points of execution?
> E.g. no nested timer events,

That's probably a good idea, yes.
Probably worth giving it a try so if/what breaks.

> never in `redisplay',

Agreed.  Feel free to consider it as a bug and hence report it when
it happens.


        Stefan