> >
> > > > IMO, licenses are to restrict the usage, not arbitrary technical
> > > > limitations. We could similary have a token declaration in FFI interface,
> > > > when
> > > > loading a library, no?
> > >
> > > You are basically reiterating what Nicolas already said, and I
> > > answered that. I see no reason to repeat my answers to these
> > > arguments, they are still the same.
> >
> > Just justifiying the statement that Emacs could have something similar as
> > GPL_COMPATIBLE token in Lisp API, just as there is in module API. I don't
> > think I have seen anyone suggesting that before, at least not what I know of.
>
> Then I don't understand what you are suggesting here. What do you
> mean by "token in Lisp API"? And if you mean something like what we
I suggest exactly that, as it sounds: a compliance token, as you have for modules.
Something like:
(defun load-library (library-name, gpl-compatible)
"Load a shared object with LIBRARY-NAME. The module has to be compatible with
GPL licence, for the deatils see https://www.gnu.org/licenses/gpl-3.0.txt, which
is asserted by GPL-COMPATIBLE been non-nil."
( ... ))
> have in the emacs-module machinery, then how is that different from
> what we already have?
It is not different at all; it is the main point and *the* reason why I suggest
it. Being in Lisp is obviously a lesser obstacle than being in C, but the same
argument(s) you presented for module code, and possible violations of the
licence, are valid for the Lisp code as well.
By the way I was about to snitch FFI from SXEmacs, but I see now Tromeys code is
in Emacs, thanks to Madhus mail. I know Tom made one, but I didn't know it was
still included in git sources.
> > > > You are preventing people not familiar with programming who can't write a
> > > > simple C wrapper to load a proprietary library, but it ain't stop any
> > > > malicious
> > > > company anyway.
> > >
> > > People not familiar with programming will be unable to use FFI for
> > > anything serious anyway.
> >
> > I think you have misunderstand what I am saying: my point was that this
> > situation is making it more tedious for experienced people to work with Emacs
> > while possibly being a stopper *only* for unexperienced programmers.
>
> I understood what you were saying, and I disagree with your assertion
> that using emacs-module API makes the job harder. If anything, it
> makes it somewhat easier: it gives the module a way to call Emacs
> primitives and provides various auxiliary facilities, without which
> adjusting a library to Emacs would be a very hard job, because Emacs
> has its own object system that is quite different from that of any
> programming language that can be used to code a module. I suggest to
> re-read the "Writing Dynamically-Loaded Modules" appendix to the ELisp
> manual and imagine how hard it would be to use FFI without many of
> those facilities provided specifically for dynamic modules.
I understand what you say as well. From my relatively short experience of
writing some FFI code for SBCL, I find it easier to work iteratively from a Lisp
repl, than writing a C code, recompile, reload and test. I can just C-x C-e in
Slime and have my glue code re-evaled and re-installed, instead of recompiling
and reloading everything.
> > However, pragmatically, I don't think putting an obstacle is the best way. On
> > the
> > contrary, I think, making it as easy as possible to use and extend Emacs for
> > whichever purpose, gives a positive incentive for people to see qualities in
> > Emacs
> > in particular, and implicitly in GNU project.
> >
> > As with every important decision, there is a balance between the gain and the
> > damage. In this case, considering how easy is to work with C libraries in some
> > other Lisp implementations, and how faster it would be to write extensions and
> > experiment with Emacs, the gain is perhaps bigger than the possible loss.
>
> There are examples out there what happens when projects take the route
> you suggest. I cannot say I'm encouraged with LLVM being used for
> proprietary closed-code extensions and targets, nor am I overwhelmed
> with the popularity of Guile, which does have unrestricted FFI. So I
> see no reason to believe what you suggest will bring such significant
> advantages to Emacs that we should seriously consider lifting the
> GPL-compatibility restriction.
I don't say you should allow unrestricted access. Just to furntiture around a
bit, and allow access to a library from Lisp, not just from C.
> > > > There is a plethora of MIT licensed math libraries, with big API surfaces,
> > > > well optimized for many architectures which users could bring into Emacs
> > > > themselves.
> > >
> > > AFAIU, there should be no reasons not to be able to load MIT licensed
> > > libraries via the emacs-module machinery.
> >
> > Nobody is stopping anyone to implement anything as modules, outside of Emacs
> > core, and I didn't say anything against modules either. I just say FFI would
> > make it easier and faster.
>
> See above: I think you are mistaken in your assessment. I don't see
> any reasons why using FFI would be so much easier than using the
> existing modules API. Given some simple boilerplate (like the DEFUN
> macro you can see in test/src/emacs-module-resources/mod-test.c, and
> other utility functions there), the job becomes quite simple. By
> contrast, using FFI the programmer would need to figure all this out
> on their own, and that requires very good familiarity with Emacs
> internals.
>
> > > For the record: there are other GNU projects that use the same
> > > "restrictions" on plugins: Gawk, GNU Make, and GCC, to mention those I
> > > know about. So it isn't like Emacs is alone in this.
> >
> > I am not sure if it is relevant with non-lisp projects.
>
> How so? GNU Make has its own scripting language, and Gawk implements
Actually two scripting lanugages. More recent one is Guile. I have never felt
need to extend make, so no idea if Guile FFI is usable from make.
> the Awk programming language, which is a full-fledged specialized
> language for data processing. How is this "not relevant" to our case?
I didn't thought it is relevant since neither make, nor awk, nor gcc are used
interactively to write Emacs applications, to solve computational problems and
similar in the way Emacs is used.
> Gawk even comes with a small set of extensions written using the
> plug-in protocol, and I invite you to review them to see how useful
> they are, even though they are relatively small and do simple tasks.
I have basically stopped using Bash & Co (tr, awk, sed etc) for any scripting. I
don't think those are bad, but as you have seen from my numerous posts on Reddit
promoting Elisp and debugging in Elisp, I do honestly find scripting in Lisp and
Emacs much more enjoyable than debugging shell scripts. I don't think it is even
comparable.
Perhaps Emanuel could write an awk implementation in Elisp, that works on Emacs
buffers, as a (one of) higher level interface. It would be probably more useful
and enjoybale for text processing than extending awk with C++ modules or
implementing some sort of DSL based on a bunch of cryptic acronyms. At least awk
is well documented. Just my opinion, I hope Emanuel is not angry on me for
reflecting and suggesting it.
By the way, a regression (sorry), perhaps someone more literally gifted than me
and better used to English language, could add a few words to Emacs manual about
how Emacs API works, regarding text processing and buffer management at
least. Something like this:
________________________________________________________________________________
Emacs Lisp uses a simple procedural model to manipulate windows, files, buffers,
and other objects.
In Emacs API, we typically manipulate a current object, whether it is a buffer,
point, window, frame, etc. It means we act on a selected buffer, selected
window, frame which we then manipulate by calling an appropriate function. Which
one we need is best looked up in the manual. The details on the calling
arguments and returned results are easiset looked up via C-h f function-name
RET.
Some of functions can take an optional object to act on, instead of acting on
the current-one, but the basic model of working with Emacs API buffer, window
and frame management, as well as text processing, is to select something (make
it current), and than acting on it.
________________________________________________________________________________
Something in that style, I don't see anything similar in introduction to the
manual, and I think it might be useful to clarify it. At least I have understand
Emacs API so.
One can also speculate how API could look like, and for illustrative purposes,
we can compare for example to similar API model like PostScript or OpenGL vs
Xlib or win32, or to iterator based design as suggested by Yuri. But I don't
think it belongs to the manual or emacs-devel list.
> > There are lots of
> > projects with different needs, some are less, others are more self-contained.
> > It
> > all depends how the software is used. Lisp(s), Emacs Lisp included, are
> > usually
> > in somewhat special situation of being relatively isolated islands when it
> > comes
> > to available software. Common Lisp perhaps being a biggest exception there.
> > An easy to use FFI makes it easier to take advantage of already existing
> > software,
> > such as mathematical libraries for example.
>
> I think you greatly underestimate the complexity of using FFI-style
> API to allow Emacs to use a mathematical library. For starters,
> floating-point values have a very specific representation in Emacs, so
> you cannot just take a value from the library and use it. Next, most
> such libraries allocate memory for its data structures (matrices
> etc.), so using them will need to provide GC glue for freeing the
> resources when an instance of a data structure is no longer needed.
> And that is just the tip of a very large iceberg.
Agree. I haven't seen Tromeys code yet, but I have seen and tried FFI in
SXEmacs, and as said in SBCL.
SBCL does not use libffi under the hood (unlike more portable CFFI), and they do
give one access to raw pointers and raw memory allocated by C runtime, as well
as the responsibility to free that allocated memory. For the good and for the
bad.
Now, Emacs also has an interface to GCC, I wonder how hard/easy is to expose C
runtime via native compiler to Lisp? I am not familiar with libgccjit, but as I
udnerstand eln files are shared objects under the hood? Does it not make the ffi
implementation easier? Sorry if it is an uninformed question, I haven't looked
that up myself.
> > Anyway, thanks for the arguments; I think it is useful to re-think the
> > arguments,
> > regardless on which side of the arguments I am.
>
> As a practical example, I suggest to take a library and integrate it
> with Emacs using the modules API, if you haven't done that already.
> It is sometimes amazing how many little details need to be figured out
> and coded for that, when the target is Emacs Lisp.
>
I have done some FFI with SBCL, and yes, I agree there are lots of details to
take care of. Error checking via Lisp is not trivial, compared to simple
if-statement in C. But still, if I f-up in SBCL with my glue, I can just rewrite
and C-x C-e in Slime and switch to repl window to test. As said above, I don't
think it is comparable to writing C code.
Från: Eli Zaretskii <eliz@gnu.org>
Skickat: den 14 augusti 2024 07:10
Till: arthur miller <arthur.miller@live.com>
Kopia: nicolas@n16f.net <nicolas@n16f.net>; emacs-devel@gnu.org <emacs-devel@gnu.org>
Ämne: Re: as for Calc and the math library
> From: arthur miller <arthur.miller@live.com>
> CC: "nicolas@n16f.net" <nicolas@n16f.net>, "emacs-devel@gnu.org"
> <emacs-devel@gnu.org>
> Date: Tue, 13 Aug 2024 21:43:00 +0000
>
> > > IMO, licenses are to restrict the usage, not arbitrary technical
> > > limitations. We could similary have a token declaration in FFI interface, when
> > > loading a library, no?
> >
> > You are basically reiterating what Nicolas already said, and I
> > answered that. I see no reason to repeat my answers to these
> > arguments, they are still the same.
>
> Just justifiying the statement that Emacs could have something similar as
> GPL_COMPATIBLE token in Lisp API, just as there is in module API. I don't
> think I have seen anyone suggesting that before, at least not what I know of.
Then I don't understand what you are suggesting here. What do you
mean by "token in Lisp API"? And if you mean something like what we
have in the emacs-module machinery, then how is that different from
what we already have?
> > > You are preventing people not familiar with programming who can't write a
> > > simple C wrapper to load a proprietary library, but it ain't stop any
> > > malicious
> > > company anyway.
> >
> > People not familiar with programming will be unable to use FFI for
> > anything serious anyway.
>
> I think you have misunderstand what I am saying: my point was that this
> situation is making it more tedious for experienced people to work with Emacs
> while possibly being a stopper *only* for unexperienced programmers.
I understood what you were saying, and I disagree with your assertion
that using emacs-module API makes the job harder. If anything, it
makes it somewhat easier: it gives the module a way to call Emacs
primitives and provides various auxiliary facilities, without which
adjusting a library to Emacs would be a very hard job, because Emacs
has its own object system that is quite different from that of any
programming language that can be used to code a module. I suggest to
re-read the "Writing Dynamically-Loaded Modules" appendix to the ELisp
manual and imagine how hard it would be to use FFI without many of
those facilities provided specifically for dynamic modules.
> However, pragmatically, I don't think putting an obstacle is the best way. On the
> contrary, I think, making it as easy as possible to use and extend Emacs for
> whichever purpose, gives a positive incentive for people to see qualities in Emacs
> in particular, and implicitly in GNU project.
>
> As with every important decision, there is a balance between the gain and the
> damage. In this case, considering how easy is to work with C libraries in some
> other Lisp implementations, and how faster it would be to write extensions and
> experiment with Emacs, the gain is perhaps bigger than the possible loss.
There are examples out there what happens when projects take the route
you suggest. I cannot say I'm encouraged with LLVM being used for
proprietary closed-code extensions and targets, nor am I overwhelmed
with the popularity of Guile, which does have unrestricted FFI. So I
see no reason to believe what you suggest will bring such significant
advantages to Emacs that we should seriously consider lifting the
GPL-compatibility restriction.
> > > There is a plethora of MIT licensed math libraries, with big API surfaces,
> > > well optimized for many architectures which users could bring into Emacs
> > > themselves.
> >
> > AFAIU, there should be no reasons not to be able to load MIT licensed
> > libraries via the emacs-module machinery.
>
> Nobody is stopping anyone to implement anything as modules, outside of Emacs
> core, and I didn't say anything against modules either. I just say FFI would
> make it easier and faster.
See above: I think you are mistaken in your assessment. I don't see
any reasons why using FFI would be so much easier than using the
existing modules API. Given some simple boilerplate (like the DEFUN
macro you can see in test/src/emacs-module-resources/mod-test.c, and
other utility functions there), the job becomes quite simple. By
contrast, using FFI the programmer would need to figure all this out
on their own, and that requires very good familiarity with Emacs
internals.
> > For the record: there are other GNU projects that use the same
> > "restrictions" on plugins: Gawk, GNU Make, and GCC, to mention those I
> > know about. So it isn't like Emacs is alone in this.
>
> I am not sure if it is relevant with non-lisp projects.
How so? GNU Make has its own scripting language, and Gawk implements
the Awk programming language, which is a full-fledged specialized
language for data processing. How is this "not relevant" to our case?
Gawk even comes with a small set of extensions written using the
plug-in protocol, and I invite you to review them to see how useful
they are, even though they are relatively small and do simple tasks.
> There are lots of
> projects with different needs, some are less, others are more self-contained. It
> all depends how the software is used. Lisp(s), Emacs Lisp included, are usually
> in somewhat special situation of being relatively isolated islands when it comes
> to available software. Common Lisp perhaps being a biggest exception there.
> An easy to use FFI makes it easier to take advantage of already existing software,
> such as mathematical libraries for example.
I think you greatly underestimate the complexity of using FFI-style
API to allow Emacs to use a mathematical library. For starters,
floating-point values have a very specific representation in Emacs, so
you cannot just take a value from the library and use it. Next, most
such libraries allocate memory for its data structures (matrices
etc.), so using them will need to provide GC glue for freeing the
resources when an instance of a data structure is no longer needed.
And that is just the tip of a very large iceberg.
> Anyway, thanks for the arguments; I think it is useful to re-think the arguments,
> regardless on which side of the arguments I am.
As a practical example, I suggest to take a library and integrate it
with Emacs using the modules API, if you haven't done that already.
It is sometimes amazing how many little details need to be figured out
and coded for that, when the target is Emacs Lisp.