non-local exits in redisplay

non-local exits in redisplay

[Akira Kyle]

Hi all,

I've been playing around with the xwidget code and I had a 
question regarding non-local exits from inside redisplay. I've 
found that if one places code somewhere in redisplay that ends up 
signaling a lisp error through xsignal, it can often cause an 
infinite redisplay loop. This seems to be due to xsignal causing a 
nonlocal exit and the next redisplay will be begin, but unless 
there's something different, the same code that caused the initial 
xsignal will execute again and hence the infinte loop. Since I do 
see code that calls lisp functions inside redisplay I was 
wondering how one should guard against such non-local exits inside 
redisplay.

As a concrete example, placing a call like

CALLN (Ffuncall, Qnil);

at the start of xwidget_init_view in xwidget.c will cause 
redisplay to infinite loop.

Note: if you actually try this you'll get a segfault which I think 
is due to xwidget-webkit-callback executing asynchronously which 
tries access state that doesn't exist yet because redisplay is 
infinite looping. To actually see the infinite loop just return 
Qnil from Fxwidget_webkit_title.

Is there a safe way to handle such cases apart from avoiding 
calling into lisp functions which may non-local exit or manually 
checking that the arguments you pass will not cause a non-local 
exit?

I'm very new to hacking on Emacs' C source code so it's also 
entirely possible I missed some bigger concept that makes this 
actually a silly question :)

Akira

Re: non-local exits in redisplay

[Stefan Monnier]

> through xsignal, it can often cause an infinite redisplay loop. This seems
> to be due to xsignal causing a nonlocal exit and the next redisplay will be
> begin, but unless there's something different, the same code that caused the
> initial xsignal will execute again and hence the infinte loop. Since I do
> see code that calls lisp functions inside redisplay I was wondering how one
> should guard against such non-local exits inside redisplay.

In `src/xdisp.c` there's things like `safe_call` (as well as
`safe_call1` and `safe_call2`) for that.

IIRC it uses the underlying condition-case machinery.


        Stefan

Re: non-local exits in redisplay

[Akira Kyle]

>
> In `src/xdisp.c` there's things like `safe_call` (as well as
> `safe_call1` and `safe_call2`) for that.
>
> IIRC it uses the underlying condition-case machinery.
>

That's exactly what I was looking for, thanks!

Now the natural follow up question: Is it ok to use the error 
functions which end up calling xsignal in redisplay when one is 
sure it won't cause this sort of infinite loop? In my testing it 
looks like the error message ends up displaying ok, but are there 
any other gotchas I should look out for here?

Akira

Re: non-local exits in redisplay

[Stefan Monnier]

> Now the natural follow up question: Is it ok to use the error functions
> which end up calling xsignal in redisplay when one is sure it won't cause
> this sort of infinite loop? In my testing it looks like the error message
> ends up displaying ok, but are there any other gotchas I should look out
> for here?

I think it's pretty risky.  IIRC there are some variables which could
remain in an incorrect state, depending on exactly when the error
is signaled.

Another way to put it is that it can be done, but depending on the
details, it may require some careful investigation to find the variables
which might need extra unwind-protect-style protection.

I think there's also a risk some redisplay may end up not being properly
drawn (e.g. because we signal the error before it's done, but it has
already be removed from the "things which need redisplay").


        Stefan

Re: non-local exits in redisplay

[Eli Zaretskii]

> From: Akira Kyle <ak@akirakyle.com>
> Date: Sun, 11 Oct 2020 18:27:13 -0600
> Cc: emacs-devel@gnu.org
> 
> > In `src/xdisp.c` there's things like `safe_call` (as well as
> > `safe_call1` and `safe_call2`) for that.
> >
> > IIRC it uses the underlying condition-case machinery.
> >
> 
> That's exactly what I was looking for, thanks!
> 
> Now the natural follow up question: Is it ok to use the error 
> functions which end up calling xsignal in redisplay when one is 
> sure it won't cause this sort of infinite loop? In my testing it 
> looks like the error message ends up displaying ok, but are there 
> any other gotchas I should look out for here?

As Stefan mentions, you are well advised not to signal errors inside
redisplay.  The reason is simple: signaling an error always reenters
redisplay immediately, because we need to display the error message.
While that is a special kind of redisplay, it usually redraws at least
one window, and can redraw more, depending on what exactly is on
display.  It is virtually impossible to predict when displaying the
error signal message will hit the same error again; even if you seem
to be able to get away with that in some situation, another user of
your code might not be so lucky, if they use some specialized feature,
like post-command-hooks that affect the display (it could be as simple
as hl-line-mode or similar), or functions that run off timers that
display something, or any number of other optional features, of which
Emacs has  a gazillion.

So you really shouldn't signal errors from redisplay.  the most you
can do is silently deposit an error message in the *Messages* buffer
(that's what the safe_call and friends do, btw) and hope that the user
will look there at some point, e.g. because the display doesn't look
"right".

Can you explain why you'd need to signal an error from the display
code?  Maybe we can find some way of dealing with your problem that is
better than just inserting a message in *Messages*.

And one more thing: please don't over-use the safe_call facility as
well, i.e. try not to call too much Lisp from the display engine.  As
convenient as that may sound, calling Lisp from display has its
downsides.  For starters, it produces more garbage, which triggers
more frequent GCs, which slows down Emacs.

So if you are tempted to call Lisp, try first to find an equivalent
way of doing that in C (don't hesitate to ask questions if you cannot
find it or are unsure how to do that), and only use Lisp if the
alternative is not reasonable.

Thanks.

Re: non-local exits in redisplay

[Akira Kyle]

On Mon, Oct 12, 2020 at 11:01 AM, Eli Zaretskii <eliz@gnu.org> 
wrote:

> As Stefan mentions, you are well advised not to signal errors 
> inside
> redisplay.  The reason is simple: signaling an error always 
> reenters
> redisplay immediately, because we need to display the error 
> message.
> While that is a special kind of redisplay, it usually redraws at 
> least
> one window, and can redraw more, depending on what exactly is on
> display.  It is virtually impossible to predict when displaying 
> the
> error signal message will hit the same error again; even if you 
> seem
> to be able to get away with that in some situation, another user 
> of
> your code might not be so lucky, if they use some specialized 
> feature,
> like post-command-hooks that affect the display (it could be as 
> simple
> as hl-line-mode or similar), or functions that run off timers 
> that
> display something, or any number of other optional features, of 
> which
> Emacs has  a gazillion.

Yes as I'm learning more about the interplay between the lisp 
interpreter and the complicated machinery of redisplay, I'm 
realizing there are many ways to shoot yoursef in the foot.

> So you really shouldn't signal errors from redisplay.  the most 
> you
> can do is silently deposit an error message in the *Messages* 
> buffer
> (that's what the safe_call and friends do, btw) and hope that 
> the user
> will look there at some point, e.g. because the display doesn't 
> look
> "right".

Which isn't very satisfactory from a UI perspective. I suppose 
setting some flag in redisplay which could then be checked later 
in lisp to signal the error would be the obvious work around.

> Can you explain why you'd need to signal an error from the 
> display
> code?  Maybe we can find some way of dealing with your problem 
> that is
> better than just inserting a message in *Messages*.
>
> And one more thing: please don't over-use the safe_call facility 
> as
> well, i.e. try not to call too much Lisp from the display 
> engine.  As
> convenient as that may sound, calling Lisp from display has its
> downsides.  For starters, it produces more garbage, which 
> triggers
> more frequent GCs, which slows down Emacs.
>
> So if you are tempted to call Lisp, try first to find an 
> equivalent
> way of doing that in C (don't hesitate to ask questions if you 
> cannot
> find it or are unsure how to do that), and only use Lisp if the
> alternative is not reasonable.

Perhaps I'm really getting ahead of myself here...
I'll describe what my goal is for the xwidgets code, which is what 
ultimately prompted this question, in a new thread in case others 
might be interested in commenting on it.

Thanks!
Akira