Native module callback when Lisp thread exits

Native module callback when Lisp thread exits

[Spencer Baugh]

Hi,

I have a native module written in OCaml which provides some useful
functions.  Sometimes, I would like to call these native module
functions from Lisp threads other than the main Lisp thread.

For a native module written in C, this works fine.  However, the OCaml
runtime requires that if it's called from other threads, the thread must
first call caml_c_thread_register; and when the thread exits, it should
call caml_c_thread_unregister. See
https://ocaml.org/manual/5.1/intfc.html#ss:c-thread-register

My native module can call caml_c_thread_register when the native module
is first called in a non-main thread; then calling my native module
functions works fine in Lisp threads.

However, there's no way for my native module to call
caml_c_thread_unregister when the thread exits.  That causes memory
leaks and means that OCaml GC performance degrades over time, since the
GC scans all known threads.

Could we add a way for a native module to call some functions when a
Lisp thread exits?

Currently, native modules expose a emacs_module_init function which is
called when the module is loaded.  Maybe a native module could
optionally also expose an emacs_module_thread_init function which is
called when a Lisp thread is created (or maybe when the module is first
used on a Lisp thread).  And likewise, an emacs_module_thread_uninit
function which is called when a Lisp thread exits.

Re: Native module callback when Lisp thread exits

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Fri, 17 May 2024 13:09:35 -0400
> Cc: mshinwell@janestreet.com
> 
> I have a native module written in OCaml which provides some useful
> functions.  Sometimes, I would like to call these native module
> functions from Lisp threads other than the main Lisp thread.
> 
> For a native module written in C, this works fine.  However, the OCaml
> runtime requires that if it's called from other threads, the thread must
> first call caml_c_thread_register; and when the thread exits, it should
> call caml_c_thread_unregister. See
> https://ocaml.org/manual/5.1/intfc.html#ss:c-thread-register
> 
> My native module can call caml_c_thread_register when the native module
> is first called in a non-main thread; then calling my native module
> functions works fine in Lisp threads.
> 
> However, there's no way for my native module to call
> caml_c_thread_unregister when the thread exits.  That causes memory
> leaks and means that OCaml GC performance degrades over time, since the
> GC scans all known threads.
> 
> Could we add a way for a native module to call some functions when a
> Lisp thread exits?
> 
> Currently, native modules expose a emacs_module_init function which is
> called when the module is loaded.  Maybe a native module could
> optionally also expose an emacs_module_thread_init function which is
> called when a Lisp thread is created (or maybe when the module is first
> used on a Lisp thread).  And likewise, an emacs_module_thread_uninit
> function which is called when a Lisp thread exits.

I don't understand why you need something module-specific.  Shouldn't
a thread-exit-hook be enough?  We could easily add that if that helps.

Re: Native module callback when Lisp thread exits

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:
> I don't understand why you need something module-specific.  Shouldn't
> a thread-exit-hook be enough?  We could easily add that if that helps.

Yes, that would be enough.

Although I'm now realizing that this could be done in my native module
by passing a destructor to pthread_key_create, so maybe I don't need
anything in Emacs after all.

Re: Native module callback when Lisp thread exits

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: emacs-devel@gnu.org,  mshinwell@janestreet.com
> Date: Sat, 18 May 2024 09:56:26 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> > I don't understand why you need something module-specific.  Shouldn't
> > a thread-exit-hook be enough?  We could easily add that if that helps.
> 
> Yes, that would be enough.
> 
> Although I'm now realizing that this could be done in my native module
> by passing a destructor to pthread_key_create, so maybe I don't need
> anything in Emacs after all.

If you don't care about portability to systems that don't use pthreads
for implementing the Lisp threads, then I guess you can do that as
well.