Re: Why is Elisp slow?

[Ergus <spacibba@aol.com>]

On Tue, May 07, 2019 at 08:51:27PM +0900, ????????? wrote:
>
>
>> 2019. 5. 7. ?????? 7:49, Ergus <spacibba@aol.com> ??????:
>>
>> Hi all:
>>
>> After reading the whole SBCL manual and the internals I have some
>> comments maybe useful if we decide to go in that direction. (If some of
>> the core developers starts with this I will be very pleased to work on
>> that, but I am too new in emacs development to start it myself alone.)
>>
>> 1) + Portability does not seems to be a big issue, they already support
>> our architectures (and some extra) and they are putting a lot of effort
>> on that. There is work to do, but in general, they already have what we
>> need from Common Lisp.
>>
>> 2) + They have documented how to port SBCL to new architectures, and some
>> of the optimizations they implement, so, if we have a compiler
>> specialist on board and finally our developers choose not to go for
>> SBCL, then maybe some of those optimization could be considered for our
>> Elisp compiler.
>>
>> 3) - There is not C binding for SBCL. To extend it, there is not a
>> public-documented API and their approach is wrapping the C functions
>> from Lisp (FFI). I wrote in their mailing list and they redirected me to
>> the CFFI page.
>>
>>   + The only advantage (I see) in this approach is that as CFFI has
>>   been ported to almost all common list compilers/interpreters; if we
>>   go in that direction for our C code we could potentially use any of
>>   those and jump from one to another at configuration time. So we
>>   don't have this problem (of being stock with a "bad" compiler)
>>   anymore.
>>
>>   - The overhead produced by wrapping in the high level instead of the
>>     low level in my experience (python, ruby, Julia) is usually much
>>     higher than having C bindings.
>>   - Writing wrappers in lisp for all our C functions exposed to Lisp I
>>     think is not an option if we don't create an automatic method. So
>>     if no one creates a plugin for sbcl to support Elisp, I really
>>     doubt the feasibility of using SBCL except if many of our current
>>     core developers agree on that and join effort for that. Else it
>>     will happen as with guile.
>
>There is an automatic method of generating C FFI bindings from header files.
>See https://github.com/rpav/cl-autowrap <https://github.com/rpav/cl-autowrap> :-)
>What I was thinking about using CL to support Elisp is to define a new
>namespace for symbols (which, in CL terms, is a so-called ???package???) named ???elisp???.
>All Cl-needed functions/macros are defined in the ???cl??? package,
>which means that you can call any cl function/macro by
>prefixing ???cl:??? in the ???elisp??? package.

This is what I understand as a plugin for SBCL. Because we will need
some steroid for that :p 

>As CL and Elisp has many common things (such as the similarity of
>many, many macros like ???defun??? or ???defvar???, etc??? or that it is a Lisp-2),
>we can implement elisp entirely inside CL itself.
>No need for developing a plugin specially for SBCL; any CL implementation will do.
>For functions defined in C, we can make or generate wrappers to get
>passed-in-arguments and dynamically wrap values into a Lisp_Obj C struct
>(AFAIU all lisp objects are represented as a struct of type info and some pointers,
> am I right?) and call the underlying function generated by cl-autowrap.
>
This is so huge that I don't know where to start looking :(. What I was
looking was a representation of the types in SBCL, but I couldn't find
that.

AFAIK cl-autowrap is to wrap simple C functions with C types and
structs, but Elisp actually has it's own representation of lisp objects
and functions. And the exposed functions already have a unified function
format to store the pointer in the function's hash table. If we use a
wrapper on the hight level we will be packing and unpacking structs
(copies and more copies) constantly and wasting the advantage of having
a similar representation. Or we won't be allowed to pass Elisp object to
CL functions.

So the first to do should be to unify the Elisp types with the CL
representation as much as possible and check the differences we can't
solve implicitly to create an explicit conversion when needed. Because
with guile, after doing the hard work, we found that there was a problem
with strings representations (if I understood right what Stefan
explained)

We also need to find a method to create lisp objects from C to implement
DEFVAR_LISP, DEFSYM and similes in C if we don't want to reimplement big
portions of code (like the display engine for example). There is still
the problem with buffer local variable that we don't have a solution for
that yet either.

Then we can start checking the functions we have that are Elisp agnostic
and can be compiled with SBCL. We could also have another set of
functions we have implemented in C we will not need anymore and finally
there will be functions we need to wrap (like everything in editfns.c
for example)... But again... this is huge.
>
>> 4) + With FFI as they implemented, it will be pretty easy to interface
>> and call C libraries and functions without needing a C wrapper to expose
>> it to lisp, so maybe an important part of our C code could be
>> substituted with Lisp (simpler and smaller) glue code. But also we will
>> be allowed to use C dynamic libraries directly without all the modules
>> complexity. (Somehow I feel that not everyone will like that if we
>> comment it in the developers mailing list :p ).
>>
>> On Mon, May 06, 2019 at 02:47:12PM -0400, Stefan Monnier wrote:
>>>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>>> need to make sure the same Emacs version can be compiled against all
>>>>> of those versions.  No idea if it would impose a significant extra
>>>>> burden or not, but it's something to be considered.  Also the fact
>>>>> that the latest release doesn't work on all those platforms is rather
>>>>> worrying.  ]
>>>>
>>>> Hmm? I can't understand :-(
>>>> Why can't Emacs can include a specific version of SBCL's source (e.g. as
>>>> a git module) and compile them all together?
>>>
>>> Exactly because the version that can run on ARM is not the same as the
>>> one that can run under Windows, which is not the same as the one that
>>> runs under AMD64.
>>>
>>> Of course, maybe I'm confused by the table of available binaries (at
>>> http://sbcl.org/platform-table.html), and in reality the latest version
>>> works fine on all supported platforms.
>>>
>>>       Stefan
>>>
>>
>