Re: dynamic reload of dynamic module not dynamic?

[Psionic K <psionik@positron.solutions>]

> but I think first we should discuss whether the FR is important enough for that

Full reload?  Absolutely indispensable and the right decision.

However, I think we're looking at the challenges wrong, worrying about
problems that could exist if dynamic modules haphazardly return
pointers to owned data rather than focusing on communicating the right
contract to dynamic module authors so that such problems don't occur.
The former conversation presumes too much responsibility on the part
of Emacs for things it can't be responsible for.

Instead of saying, "what if the dynamic module does this?" where
"this" would be hard for Emacs to manage in the next five or ten
years, tell dynamic module authors to conform to a contract that is
simple for Emacs already.  That way we're not all blocking each other.

Just looking at the Rust module code, I see a mechanism to give the
Emacs GC ownership of a value.  If that can be done, why would I hold
in Elisp any pointers into memory managed by the dynamic module?  When
we do want to name and point to things in the dynamic module, we
should do so by identifier, not a raw pointer.  That way the dynamic
module can look before leaping, using whatever identifier mapping the
module author sees fit.

Any loss in speed of not handing pointers over to Emacs is silly to
worry about because the presumption of the entire scheme is that Elisp
is delegating human speed command input to a fast and capable
implementation.  Lookups for identifiers are faster than human speed,
so the cost of looking up identifiers at the human to native interface
is negligible.  There was a keystroke.  We shouldn't be counting
microseconds.  The presumption is that we saved ten seconds of Elisp
or did something Elisp can't do well.

From the rest of the thread and in general, I got the impression that
Emacs devs are, out of experience with users making new problems for
themselves, being too protective in thinking.  Dynamic module authors
should be held to a higher standard and not treated like a potential
source of annoying bug reports.  Just set some ground rules that will
make upstream behavior within Emacs easier to plan on rather than
presuming too much responsibility.  The user is writing a dynamic
module.  They have opted into knowing what they are doing.  There
should minimally only be ways made available to write a good module.
The contract should be that Emacs can reload the module and that it's
the module's responsibility not to violate that contract by handing
back data that won't work through reload.  Emacs should treat them
like oracles that are responsible for not screwing up.

I think this is the right conversation to have.  It's a C interface.
The user is responsible.  Honestly it is more of a headache to
consider how to tell Emacs to invalidate raw pointers than to manage
the data correctly within the module.  There's infinite other ways to
crash programs that dynamic module authors are also responsible for.
It's nothing new.  Delegate this responsibility to the maximum extent
possible and dynamic module authors will figure it out with libraries.

The alternative is to develop by throwing away Emacs instances or load
lots of renamed and annoying duplicate modules?  A lot of Elisp state
might be built up as the user is hacking away with a module and
associated Elisp concurrently.  When forced to reload Emacs, it will
just break the workflow.  This is not a 1.0 version of the dynamic
module feature because it negates the core strength of interactive
dynamic development.

In the end, the worst case of badly behaved dynamic modules had nearly
identical impact to the user: reload Emacs (if it didn't politely
crash already).

Thanks