Re: emacs-dynamic-module in Emacs Git?

[Eli Zaretskii <eliz@gnu.org>]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Wed, 03 Dec 2014 04:04:40 -0600
> 
> >> > I don't think this is correct: we don't really want to export all the
> >> > symbols.
> >> 
> >> Why not?
> >
> > Security: you don't want to expose all of the Emacs bowels to any
> > external program out there.
> 
> There are many other aspects to security; I doubt this particular
> strategy will really help.
> 
> There are better ways to prevent bad code getting into Emacs; code
> reviewed signed modules is probably the best way.

See David's response, with which I fully agree.  I'm sure you had your
share of vulnerabilities exploited by bugs if not by malicious
software, and you know very well the dangers of such over-exposure.

> That's essentially how we currently prevent bad code in Emacs core.

How can we code-review modules that are not bundled?  We have no
control on those whatsoever.

> I'm advocating allowing any code that could be in Emacs core to instead
> by in a dynamic module, to allow separate development subject to the
> same restrictions as Emacs core code - FSF copyright, code review.
>
> Obviously, the ability to load dynamic modules allows users to choose
> other modules that do not meet those criteria. But that should not
> restrict what we can do in a dynamic module. We are _not_ building a
> sandbox, but a powerful development environment; people must be allowed
> to shoot themselves in the foot.

I'm okay with allowing people to shoot themselves, but I'm not okay
with letting them shoot Emacs.

What you suggest is a very slippery slope.  If we agree to such an
unlimited exposure of internals, I'm quite sure that before long we'll
have modules all over the place depending on those internals, and
their authors will apply pressure not to change the internals on which
they happen to depend.  I don't think we want Emacs development to
become hostage to every package out there.  No other project I know
of, not even libraries (whose proclaimed raison d'être is to expose
APIs) do that, and for very good reasons.  I see no reason why Emacs,
of all the packages, should do what no other one does.

> > Or ask yourself why the latest GCC and Binutils default to not export
> > anything, contrary to what they did in older versions.
> 
> I would guess that has more to do with namespace control, but I'd have
> to read the rationale to be sure.

Yes, that too.  Won't there be problems in that department as well,
e.g., if some library function replaced by Emacs clashes with its
namesake in the module?  With toy modules, this is not a problem, but
what about those that will use large libraries, where it's not so easy
to rename a function?  Again, very slippery slope.

> Since we are talking about code that is intended to be tightly
> integrated with Emacs, we _want_ the Emacs namespace to be visible.

I'm not sure who is "we" here.  I don't think I've heard Stefan, or
anyone else expressing such far-reaching desires.

> >> If we were writing this code to be included in Emacs core, we'd have
> >> access to all of the symbols.
> >
> > You can have access to symbols via specific protocols without
> > exporting everything.  And I'm not sure you really do need access to
> > everything.
> 
> Protocols tend to get in the way. If a pipe interface was viable, I'd
> use it. But it's not; I need direct, tight integration in order to be
> fast enough.

A software API is much faster than a pipe, so I don't see how this
comparison is useful, or even valid.

> I am very sure I don't need access to absolutely every symbol in the
> Emacs namespace. But it's not worth the time to try to figure out ahead
> of time which ones I might need. I can certainly provide a list of what
> was used after I've got a first version working.
> 
> Stefan's approach makes sense; try to define an API, but assume it will
> change/evolve. I'm simply arguing that it will not be worth the effort.
> The only way to find out is to try it.

Trying is fine, but it's a two-way street: expect the maintainers to
resist adding some of the symbols to your list.  There will be
negotiations, and at least I will object to granting access to
everything, which I consider insane in the long run.  Every interface
and every bit of internals we expose should be scrutinized to
determine whether exposing it is a good idea, how necessary that is,
what are its chances to change without notice, etc.

> >> _if_ the module author wants to be somewhat isolated from Emacs changes,
> >> and/or support more than one Emacs version, then they will want to stick
> >> to some stable subset. But I don't think we can define that subset ahead
> >> of time, and it will certainly be a different subset for each module.
> >> 
> >> We don't have a single .el file that defines the "Emacs core elisp API
> >> for packages"; I don't think we can define a .h file for modules either.
> >
> > You are just re-iterating my doubts about usability and wisdom in this
> > feature.
> 
> I don't understand.
> 
> You say it would be ok to add this code to core Emacs; all of the
> statements above would apply to that choice as well.

When code is added to the core, it is automatically updated when the
internals change.  That's the huge difference you are overlooking.

> We are talking about a dynamically linked module in a separate source
> repository, as compared to a staticly linked one in the core repository.
> Why should that choice affect the choice of the namespace that is
> visible to the module?

I hope by now you understand my answer to that question.

> >> Let's make it simple; export all of them.
> >
> > The other alternative is also simple; see GNU Make for an example.
> 
> By "the other alternative" I assume you mean "define an Emacs module
> API"

Not necessarily.  It should be enough to make a list of interfaces to
which we allow external linkage.

> that's _not_ simple. Proof: no one has done it yet, but "export
> all of them" has been done. QED.

:-)

Do take a look at GNU Make, though.  It might be useful to put our
case in context and in its true proportions.

> Note that there are actually two namespaces we are talking about here;
> the compile time namespace, determined by .h files, and the link time
> namespace, determined by --export-dynamic and/or link libraries.
> 
> The future maintenance issue is best addressed via .h files; don't put
> functions you don't want to support in future versions in any .h file,
> or have a naming convention that clearly indicates which .h files will
> be supported.

If we make the effort to create the header file, restricting the
linkage only to those interfaces is a trivial job.

> I don't see any reason to restrict the link time namespace.

It makes no sense whatsoever to allow linkage to interfaces that are
not declared in the header file.  Do you expect package authors to
reverse-engineer or disassemble Emacs in order to find that info?