Re: Dynamic modules: MODULE_HANDLE_SIGNALS etc.

[Eli Zaretskii <eliz@gnu.org>]

> Cc: aurelien.aptel+emacs@gmail.com, p.stephani2@gmail.com, dancol@dancol.org,
>  tzz@lifelogs.com, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 22 Dec 2015 15:28:03 -0800
> 
>     Is that practical?  One problem that I, as a module developer would
>     face, is to estimate the amount of stack I'd need for unwinding.
>     Where do I begin?
> 
> We could merely require that any module needing recursion must call a new stack-size-checking function at each level of recursion, in order to detect stack overflow.

That's performance penalty.  I don't think we should incur that, on
behalf of a problem that really should "never" happen.

> Also, any module with an unbounded amount of computation should call the equivalent of QUIT every now and then. If the module API doesn't let (or ideally, require) modules to do that now, it should. Otherwise it's an Emacs freeze waiting to happen.

I agree, but I think it's unrelated to stack overflow.  (Though I
think we will need to have a module-specific version of QUIT, as long
as we keep the constraint of not directly longjmping from module
code.)

>     Then there are all the stack frames below the module, which belong to
>     some Lisp someone else wrote -- who will be responsible for ensuring
>     those other unwinders don't need large amounts of stack space that
>     might be unavailable at stack-overflow point?
> 
> Each stack-overflow unwinder should use only a small amount of stack space, in order to prevent an infinite loop. We could partially enforce this by detecting trivial infinite stack-overflow loops.

I don't think it's practical to require that unwinders use "only a
small amount of stack space".  Verifying that is non-trivial even if
you know how the particular internals work; it's downright impossible
if you are a Lisp programmer with only a minimal knowledge of C and
the Emacs C levels, or none at all.

>     Stack overflow detection on modern systems uses hardware assistance
>     and processor exceptions to detect overflow with no runtime
>     penalties.  Doing the equivalent in application code is bound to incur
>     additional processing, which will slow down code, right?
> 
> Yes, but it's not a big deal.

I disagree.  Compilers' settings to allow run-time stack-smashing
detection are off by default for a good reason.

>     If you think
>     about manipulating the guard pages to make them resizable, are we sure
>     enough of the supported platforms allow that?
> 
> In platforms that don't support guard pages we'll need to have the run-time checks. Either approach should be adequate.

I didn't mean platforms that don't support guard pages.  I meant
platforms that do, but don't allow you to mess with them.  For
starters, you need to know the address of the page at the end of the
stack, which is not trivial.  Then you need to change its protection,
which I'm not sure all platforms will let you do, certainly in
future-proof manner.  (Don't modern hardened system prevent messing
with page protection bits?)

>     Hitting stack overflow when some module runs is even rarer.
>     Why is it a disaster to fail to invoke the unwinders in those cases?
> 
> Often I expect it wouldn't be a disaster; it'd just be a memory leak. I suppose there could be some modules where it would be a disaster. But perhaps we can just ask people to not write such modules, by making it a part of the Emacs API that unwinders might not be invoked. 

I think if we adopt the view that the user should save and exit ASAP
following recovery from stack overflow (similar to what we expect when
we run out of memory), the probability of a "disaster" in this
scenario becomes much lower, perhaps negligible.

All in all, I think the current recovery is "good enough", and
investing significant efforts into making it slightly better will
waste resources that are better applied to more important problems.