Re: Dynamic loading progress

["Aurélien Aptel" <aurelien.aptel+emacs@gmail.com>]

Hi all, Daniel,

Following Ted last off-list email I looked again at the module code.

Thanks again to Daniel for the API design proposal you did few month
ago and the general explanations, it was insightful and it really
helped me. I would really like to have the module API in emacs 25 so I
recently revisited my code and re-read the dynamic loading threads on
emacs-devel but I'm still stuck on the memory aspect.

I'll re-explain briefly the problem:

The module API [1] lets module writer make new Lisp_Object (hidden as
opaque pointers named 'emacs_value') via make_<type>(), funcall(),
type_of() and intern() functions. But if a module function calls core
Emacs functions, those functions it turn might call the GC, which
would free and invalidate the objects made by the module.

Here are 2 examples of the same problem:

- Local values which are created/can-be-freed inside the module function.

    a = make_string("foo")
    b = make_string("bar")
    len = funcall(eval, 1, [length_sym, b])
    // a (and b?) might be invalid now if the GC was triggered inside
the funcall call.

In the core Emacs code, we have the GCPRO macro to deal with that use-case.

- Values created and reused between 2 module calls.

    emacs_value global_a;

    module_func_A()
        global_a = make_string("a")

    module_func_B():
        // global_a might be invalid now if the GC was triggered
between func_A and func_B
        return global_a

In the core Emacs code, global_a is usually a defvar.

Daniel was talking about tables mapping emacs_value to Lisp_Object,
and having manual memory management in the module API. But I'm not
sure I understood what you meant.

Here's another proposition (which might just be what you were trying
to explain):

We add a hash table at the GC root that maps module names to a list of
values currently owned by the module.
Any markable/collectable emacs_value (all Lisp_Object types except for
numeric ones basically) created by the module API are added to that
hash table (appended at the module's list).
When a module doesn't need a value anymore, it explicitly calls a
function in the module API that removes it from the module's list.
The GC walks the object tree like usual, including the module hash table.

This is more or less what GCPRO/UNGCPRO does, I think (temporally add
objects to the GC root). And if remember correctly Stefan was against
exposing it in the module API.

I've proposed a hash table to know which values a module owns. That
way we can sort of "unload" a module by simply removing the module
entry from the hash table (we still have to deal with unbinding the
module's functions). The GC will do the rest.

Any feedback, other ideas, welcome.

1: https://github.com/aaptel/emacs-dynamic-module/blob/dynamic-modules-2/src/emacs_module.h