Re: Dynamic loading progress

[Philipp Stephani <p.stephani2@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6513 bytes --]

Eli Zaretskii <eliz@gnu.org> schrieb am Sa., 21. Nov. 2015 um 09:35 Uhr:

> > From: Philipp Stephani <p.stephani2@gmail.com>
> > Date: Fri, 20 Nov 2015 23:22:49 +0000
> > Cc: tzz@lifelogs.com, aurelien.aptel+emacs@gmail.com,
> emacs-devel@gnu.org
> >
> >     Ah, you are talking about C++ dynamic initializers! So the model is
> >     that someone writes a module in C++, starts a thread there, and then
> >     inside some dynamic initializer calls emacs-module interface
> >     functions, is that it? If that's the situation, I'd suggest a
> >     prominent commentary describing this in the source.
> >
> > The SO post talks about C++ but the issue is the same in C. AFAIK with
> C11 and
> > C++11 the execution models are harmonized.
>
> It doesn't matter for the issue at hand whether it's C, C++, Java, or
> whatever.  My originally incorrect interpretation of what you wrote
> was that you are talking about initializers that are part of the
> module code, i.e. the emacs_module_init function they implement.
>
> > It seems the comment is overly confusing. It is supposed to warn about
> the
> > following. Naively, if you wanted to test whether you are in the main
> thread,
> > you would do (module types and naming):
> >
> > static thread_id main_thread = get_current_thread();
> > bool in_main_thread() { return get_current_thread() == main_thread; }
> >
> > The dynamic initializer here is the first "get_current_thread()"; it is
> not
> > guaranteed to run in the main thread, so "main_thread" is not guaranteed
> to
> > contain the main thread ID.  Therefore you have to do:
> >
> > static thread_id main_thread; // initialized later
> > int main() {
> > // guaranteed to be in the main thread
> > main_thread = get_current_thread();
> > }
> >
> > That's all. I'm not aware of any runtime that would run dynamic
> initializers
> > outside of the main thread, but it's not impossible and easy to protect
> > against.
>
> AFAIK, code that does this:
>
>   static thread_id main_thread = get_current_thread();
>
> is invalid in C, because such initializers cannot call functions.  So
> it would surprise me to see code which tried to record its thread ID
> before 'main'.
>

Indeed, you're right. (Yes, I'm a C++ programmer :))
Sorry for the confusion.


>
> So I think we should reword that comment to be much less mysterious
> and confusing than it is now.  (Look how much did we need to talk
> about for you to explain to me what was the intent of the comment.)
>

Since the alternative isn't even legal, the comment can just go away.


>
> >     Anyway, thanks for explaining this, I now know how to change the code
> >     to DTRT on MS-Windows wrt to the thread checks.
> >
> > This is unfortunately all surprisingly subtle and vaguely defined. See
> e.g.
> > http://stackoverflow.com/q/19744250/178761 (apparently the standards
> are vague
> > about what happens to detached threads after main has exited).
>
> I don't see how that affects the issue at hand.  The issue at hand is
> whether a thread ID of the main thread could be reused while some of
> the other threads belonging to the Emacs process are still running.
> And the answer to that on MS-Windows is AFAIU a sound NO, because as
> long as the Emacs process is alive, it holds a handle on the main
> thread, which precludes the OS from discarding that thread's kernel
> object.  Why? because a thread handle can and is used to query the OS
> about that thread's conditions, like its exit code, or wait for its
> completion in the likes of WaitForSingleObject.  So the kernel object
> that represents the thread must be kept by the OS as long as at least
> one open handle for the thread exists, and that prevents the OS from
> reusing the thread ID.
>

Does it actually hold that handle? It sounds reasonable, but I can't find
it documented.


>
> >     > See also
> >     > http://blogs.msdn.com/b/oldnewthing/archive/2006/09/27/773741.aspx
> .
> >
> >     We don't use IsBadWritePtr on Windows to check this, see
> >     w32_valid_pointer_p for how this is actually implemented.
> >
> > Much of this applies generally.
> > > "But what should I do, then, if somebody passes me a bad pointer?"
> > > You should crash.
>
> Which is what we do, since eassert aborts.  We will just do it sooner,
> which IME is a Good Thing.
>

OK. We should be a bit careful with the current implementation of
valid_pointer_p though, as AFAIK write(2) could cast the pointer it
receives to char* and dereference it, but that's probably no worse than not
checking it at all, and unrelated to modules.


>
> >     Anyway, I'm surprised by this extreme POV: even if we cannot validate
> >     a pointer 100%, surely it doesn't mean we cannot or shouldn't do some
> >     partial job? Why this "all or nothing" approach?
> >
> > We can check whether it's NULL. Apart from that, everything else is
> outside of
> > the C standard.
>
> Emacs is not a Standard C program, far from it.  It uses a lot of
> stuff outside of any C standard, and for a very good reason: it is a
> large and complicate program with many features that require access to
> OS facilities.
>
> IOW, using only Standard C features is not, and cannot be, a
> requirement for Emacs code.
>

Fair enough.


>
> >     We need to devise a way for it to detect that it was called from
> >     emacs-module.c, the rest is simple, I think.
> >
> > Hmm, why does it need to detect anything? Can't it just be a different
> function
> > that doesn't signal, similar to push_handler and push_handler_nosignal?
>
> I don't think we want each of its callers call the signaling part by
> itself.  That would be repeating the problem with malloc itself: many
> programs simply neglect to include the code which does TRT when it
> returns NULL.  xmalloc solves this, and makes sure the (non-trivial)
> error action and message are always the same in that case.
>
> We need a variant of this for memory allocated on behalf of modules, I
> think.
>

But this would require modules to be prepared for handling longjmps, which
in general they aren't.
In an "unsafe" language like C, we can't do without the users' cooperation.
If users ignore NULL returns from malloc, that's a bug and we can't do much
about it. We could make it harder to accidentially ignore the result:
bool emacs_malloc(size_t size, void **out)
__attribute__((warn_unused_result));
but that requires a compiler extension and is not really consistent with
the rest of the module interface, where NULL is regularly returned on
failure.

[-- Attachment #2: Type: text/html, Size: 8752 bytes --]