Three strikes them out

Three strikes them out

[Lennart Borgman (gmail)]

Sometimes I am able to get fontification to loop. That may lock Emacs 
totally.

It would be good to have some way to stop timers from running in such 
situations - if it helps, I am not sure. Maybe three C-g in a row

   C-g C-g C-g

would be a good convention for stopping all timers (and not let them 
start again)?

But I am not sure if it is possible to implement something that can stop 
the timers. Is it possible?

Re: Three strikes them out

[Stefan Monnier]

> Sometimes I am able to get fontification to loop. That may lock
> Emacs totally.

> It would be good to have some way to stop timers from running in such
> situations - if it helps, I am not sure. Maybe three C-g in a row

What makes you think the problem is in the timers?
I can't remember encountering such a problem.


        Stefan

Re: Three strikes them out

[Lennart Borgman (gmail)]

Stefan Monnier wrote:
>> Sometimes I am able to get fontification to loop. That may lock
>> Emacs totally.
> 
>> It would be good to have some way to stop timers from running in such
>> situations - if it helps, I am not sure. Maybe three C-g in a row
> 
> What makes you think the problem is in the timers?
> I can't remember encountering such a problem.

I know that something is looping during fontification. (Sometimes it is 
my bad, sometimes it is things I do not understand.)

That is all I know.

Re: Three strikes them out

[martin rudalics]

 >> What makes you think the problem is in the timers?
 >> I can't remember encountering such a problem.
 >
 > I know that something is looping during fontification. (Sometimes it is
 > my bad, sometimes it is things I do not understand.)

What happens when you set `jit-lock-context-time' to a very large value?

Re: Three strikes them out

[Lennart Borgman (gmail)]

martin rudalics wrote:
>  >> What makes you think the problem is in the timers?
>  >> I can't remember encountering such a problem.
>  >
>  > I know that something is looping during fontification. (Sometimes it is
>  > my bad, sometimes it is things I do not understand.)
> 
> What happens when you set `jit-lock-context-time' to a very large value?

I can dig up one of those cases that loops and try it, but why do you 
propose that? (It might of course help a bit to find out where the 
problem happens and I have one case where this could indeed be 
interesting, but I am not sure if that is what you mean.)

The general problem that looping in a timer may hang Emacs is what I 
really wanted to address here.

Re: Three strikes them out

[martin rudalics]

 > What happens when you set `jit-lock-context-time' to a very large value?
 >
 > I can dig up one of those cases that loops and try it, but why do you
 > propose that? (It might of course help a bit to find out where the
 > problem happens and I have one case where this could indeed be
 > interesting, but I am not sure if that is what you mean.)

If font-lock loops within contextual refontification there's either a
bug in font-lock (which I consider highly impropable) or there's a bug
in your code (maybe some function called by a hook or another timer)
that changes the buffer and re-triggers contextual refontification.
Whatever applies should be resolved.

 > The general problem that looping in a timer may hang Emacs is what I
 > really wanted to address here.

You can't: As soon as you manage to exit looping, a repeating idle timer
will fire again.

Re: Three strikes them out

[Lennart Borgman (gmail)]

martin rudalics wrote:
>  > What happens when you set `jit-lock-context-time' to a very large value?
>  >
>  > I can dig up one of those cases that loops and try it, but why do you
>  > propose that? (It might of course help a bit to find out where the
>  > problem happens and I have one case where this could indeed be
>  > interesting, but I am not sure if that is what you mean.)
> 
> If font-lock loops within contextual refontification there's either a
> bug in font-lock (which I consider highly impropable) or there's a bug
> in your code (maybe some function called by a hook or another timer)
> that changes the buffer and re-triggers contextual refontification.
> Whatever applies should be resolved.

Thanks. The situation is a bit different in my case, but it could 
perhaps be that contextual refontificaion is re-triggered. Could that 
happen when the code runs under font-lock and only text properties and 
overlays are changed?

>  > The general problem that looping in a timer may hang Emacs is what I
>  > really wanted to address here.
> 
> You can't: As soon as you manage to exit looping, a repeating idle timer
> will fire again.

Yes, that was why I suggested that we implement something that allows us 
to stop this (with for example C-g C-g C-g).

Re: Three strikes them out

[martin rudalics]

 > Thanks. The situation is a bit different in my case, but it could
 > perhaps be that contextual refontificaion is re-triggered. Could that
 > happen when the code runs under font-lock and only text properties and
 > overlays are changed?

It happens with every change of a text property.

 >> You can't: As soon as you manage to exit looping, a repeating idle timer
 >> will fire again.
 >
 > Yes, that was why I suggested that we implement something that allows us
 > to stop this (with for example C-g C-g C-g).

Stop what?  All idle timers?  That's hardly possible if the timer fires
before you finished typing that key sequence.  If, on the other hand,
you manage earlier keyboard events get through you will stop all timers
whenever you mishappen to type C-g repeatedly.

If there's a bug fix the bug.  If the bug is caused by a function run by
a timer make the delay larger or run the function manually to find the
bug.  In general any function run by a timer must be safe - that is exit
quickly wrt real time and never cause any reentrancy.

Re: Three strikes them out

[Lennart Borgman (gmail)]

martin rudalics wrote:
>  > Thanks. The situation is a bit different in my case, but it could
>  > perhaps be that contextual refontificaion is re-triggered. Could that
>  > happen when the code runs under font-lock and only text properties and
>  > overlays are changed?
> 
> It happens with every change of a text property.


Does that mean I have to arrange to avoid after-change-functions to be 
run during fontification?

Re: Three strikes them out

[joakim]

martin rudalics <rudalics@gmx.at> writes:
>
> Stop what?  All idle timers?  That's hardly possible if the timer fires
> before you finished typing that key sequence.  If, on the other hand,
> you manage earlier keyboard events get through you will stop all timers
> whenever you mishappen to type C-g repeatedly.
>
> If there's a bug fix the bug.  If the bug is caused by a function run by
> a timer make the delay larger or run the function manually to find the
> bug.  In general any function run by a timer must be safe - that is exit
> quickly wrt real time and never cause any reentrancy.

I agree with Lennart that Emacs should provide more facilities to get
out of awkward situations. Sure, fix the bug, but how is any given Emacs
user affected by some Emacs bug somewhere supposed to do that?

Lennarts "c-g repeatedly" maybe wasnt the best example. Surely this
doesnt exclude the possibility of adding some other helpful feature?

I, for example, would like a facility to attach GDB on a keystroke, and
some nice(r) gdb macros to examine why emacs is ill. But thats me.

-- 
Joakim Verona

Re: Three strikes them out

[martin rudalics]

 > Does that mean I have to arrange to avoid after-change-functions to be
 > run during fontification?

I'm afraid I don't understand your question.  Font-lock routines bind
`inhibit-modification-hooks' to t so these won't be run during
fontification.  I suppose you use some other hook or timer based code
that modifes the buffer and triggers contextual fontification over and
over.

Re: Three strikes them out

[martin rudalics]

 > I agree with Lennart that Emacs should provide more facilities to get
 > out of awkward situations. Sure, fix the bug, but how is any given Emacs
 > user affected by some Emacs bug somewhere supposed to do that?

If the bug is with Emacs' fontification routines there's hardly anything
a user can do but give a simple and precise recipe to reproduce the bug
and wait until someone familiar with the code fixes it.  At least that's
what I would do.

 > Lennarts "c-g repeatedly" maybe wasnt the best example. Surely this
 > doesnt exclude the possibility of adding some other helpful feature?

If it's timer based it might depend on whether you're able to intercept
it.

Re: Three strikes them out

[Lennart Borgman (gmail)]

martin rudalics wrote:
>  > Does that mean I have to arrange to avoid after-change-functions to be
>  > run during fontification?
> 
> I'm afraid I don't understand your question.  Font-lock routines bind
> `inhibit-modification-hooks' to t so these won't be run during
> fontification.

Ah, thanks. That was not clear to me.

 > I suppose you use some other hook or timer based code
> that modifes the buffer and triggers contextual fontification over and
> over.

This time I happened to change the text property 'fontified (because of 
a test I forgot to remove). But the hang I experienced during this did 
not lock up Emacs totally.

Thinking about it again that is what I expect since the loops involved 
restarting idle timers.

Re: Three strikes them out

[Lennart Borgman (gmail)]

martin rudalics wrote:
>  > I agree with Lennart that Emacs should provide more facilities to get
>  > out of awkward situations. Sure, fix the bug, but how is any given Emacs
>  > user affected by some Emacs bug somewhere supposed to do that?
> 
> If the bug is with Emacs' fontification routines there's hardly anything
> a user can do but give a simple and precise recipe to reproduce the bug
> and wait until someone familiar with the code fixes it.  At least that's
> what I would do.

And it is still very hard to find those bugs sometimes. One suggestion 
that have been given here to test fontification code is to run it by 
calling for example font-lock-fontify-region.

That does not always work because that function fontifies from top to 
bottom, but the bug may perhaps only show up in other situations.

I think it would be useful to be able to set `debugger' to something 
that can log an error during fontification. That is currently not possible.


>  > Lennarts "c-g repeatedly" maybe wasnt the best example. Surely this
>  > doesnt exclude the possibility of adding some other helpful feature?
> 
> If it's timer based it might depend on whether you're able to intercept
> it.

I just wanted a C-g signal to reach the timers just as it now reaches 
code running directly in the command loop. I thought that some C-g in a 
row would be easy to remember for that. (Of course some more things 
needs to be stopped for this to be meaningful. Timers should be stopped 
from reenabling themselves etc.)

Re: Three strikes them out

[Miles Bader]

"Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:
> I just wanted a C-g signal to reach the timers just as it now reaches
> code running directly in the command loop. I thought that some C-g in a
> row would be easy to remember for that.

I think that would an _extremely_ bad idea -- I often hit C-g like crazy
when response is just the tiniest bit slow (or even simply because I'm
angry at something else).

If parts of emacs just stopped working, in invisible and hard to debug
ways, just because the user happened to hit C-g a few too many times, it
would be very annoying and confusing for users.

-Miles

-- 
Opportunity, n. A favorable occasion for grasping a disappointment.

Re: Three strikes them out

[Lennart Borgman (gmail)]

Miles Bader wrote:
> "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:
>> I just wanted a C-g signal to reach the timers just as it now reaches
>> code running directly in the command loop. I thought that some C-g in a
>> row would be easy to remember for that.
> 
> I think that would an _extremely_ bad idea -- I often hit C-g like crazy
> when response is just the tiniest bit slow (or even simply because I'm
> angry at something else).

I would rather say it would be an extremely bad idea to hit C-g like 
that if my idea was implemented ;-)

> If parts of emacs just stopped working, in invisible and hard to debug
> ways, just because the user happened to hit C-g a few too many times, it
> would be very annoying and confusing for users.

Indeed, but still I think it would be useful to be able to do just that 
sometimes. With another key combination and a descent message explaining 
the state ...

Re: Three strikes them out

[martin rudalics]

 > And it is still very hard to find those bugs sometimes. One suggestion
 > that have been given here to test fontification code is to run it by
 > calling for example font-lock-fontify-region.
 >
 > That does not always work because that function fontifies from top to
 > bottom, but the bug may perhaps only show up in other situations.

You could try writing a small testbed which turns off regular font-lock
and allows to _manually_ trigger refontification (and contextual
refontification) according to what jit-lock and font-lock would do in
case of a buffer change.  Then you could step through these routines
with edebug without being hit by intercepting timers.

Re: Three strikes them out

[Lennart Borgman (gmail)]

martin rudalics wrote:
>  > And it is still very hard to find those bugs sometimes. One suggestion
>  > that have been given here to test fontification code is to run it by
>  > calling for example font-lock-fontify-region.
>  >
>  > That does not always work because that function fontifies from top to
>  > bottom, but the bug may perhaps only show up in other situations.
> 
> You could try writing a small testbed which turns off regular font-lock
> and allows to _manually_ trigger refontification (and contextual
> refontification) according to what jit-lock and font-lock would do in
> case of a buffer change.  Then you could step through these routines
> with edebug without being hit by intercepting timers.

It is a good idea. I could extend my testbed like that.

However if a user discovers the problem it would be good to have a 
debugger that writes a log somewhere that a user can send with a bug report.

Re: Three strikes them out

[Stefan Monnier]

>> >> What makes you think the problem is in the timers?
>> >> I can't remember encountering such a problem.
>> >
>> > I know that something is looping during fontification. (Sometimes it is
>> > my bad, sometimes it is things I do not understand.)
>> 
>> What happens when you set `jit-lock-context-time' to a very large value?

> I can dig up one of those cases that loops and try it, but why do you
> propose that? (It might of course help a bit to find out where the problem
> happens and I have one case where this could indeed be interesting, but I am
> not sure if that is what you mean.)

> The general problem that looping in a timer may hang Emacs is what I really
> wanted to address here.

Yes, but depending on how it happens, the way to solve it may
be different.  E.g. is the problem that C-g is ignored?  Or that it is
obeyed but we immediately get back into the problematic code?
Is the problematic code really triggered by timers, rather than by
display hooks like jit-lock?

I think there's a lot of room for improvement in how Emacs allows the
user to get back control in some cases, but this is all very difficult.
Regarding Miles's issue with "hitting C-g frantically", this is an
important point (I do that too in all kinds of cases), but that doesn't
mean we can't do what you want: it just means we have to distinguish
those two situations.  E.g. the "3 C-g" rules might only apply if
some amount of time elapses between them, and if the currently running
code in all 3 is "similar", ... that doesn't sound quite right either,
but that just indicates that it's a hard problem ;-)


        Stefan