Re: Why is Elisp slow?

[Ergus <spacibba@aol.com>]

Hi:

After some mails in the SBCL mailing list and make some
question-reply-question there; I arrived the conclusion that SBCL is not
an alternative :( I was very wrong.

I'll copy-paste the main answers here just in case someone wants to go
this way in the future. Or in case someone more expert see some possible
workaround for the issues exposed here.

----------------------------------------

Your question as stated is too open-ended for me to discern the exact
intent, but I think's it's something like this: there is much common-lisp
code that people run in emacs, and you'd like not to have to maintain a
common-lisp compiler as well as elisp compiler. Instead you'd like to
retarget the output of the SBCL compiler to produce elisp byte code for
execution.
It's unclear whether you'd expect this to be an out-of-process compiler or
an in-process compiler.  If in-process, then you've got larger problems
than figuring out how to call C.  SBCL is not intended to be a "subsystem"
of other lisps, nor even to allow main() to be anything other than its own
main(), though that limitation has been relaxed a bit.

If out-of-process, then I would imagine that SBCL would operate as a file
compiler only, and not a compiler of arbitrary forms. You need to figure
out how to treat emacs as the virtual machine which the SBCL compiler
targets, and a suitable interface for getting the compiled code out of an
SBCL '.fasl' file and into emacs.   The "machine code" in the '.fasl' file
would actually be emacs byte code. (I'm assuming that you plan to keep
more-or-less the same byte code execution engine)

I don't see why C functions are a particular problem.  SBCL uses C
functions all the time.  In fact it is extremely interoperable with C, but
that's really asking the wrong question imho.
Portability is the real question, since you brought it up. Are you implying
that SBCL's compiler should produce machine code that would run within
emacs? This is highly unlikely; I won't say impossible, but darn near so.
 First of all, SBCL can produce machine code only for a tiny fraction of
the CPUs that emacs supports. Secondly, producing machine code assumes that
the entirety of the runtime (comprised of the memory layout, heap objects,
and support routines) is exactly as SBCL expects it to be - symbols look
like SBCL symbols, special variables get bound in exactly the way that SBCL
expects them to get bound.  SBCL's compiled lisp files are not a portable
abstraction like a '.o' file (or '.so' if you like) that could be loaded
into *any* other program (a C program, Java program, etc) and just run.

But as I said initially, I think you're trying to suggest that you would
produce code that is executed by the emacs lisp engine, not the CPU
directly.
So that would be essentially 1 new architecture that SBCL would have to
support, namely the emacs virtual machine.  That sounds more plausible and
there's no reason to care about the set of CPUs that we can directly
compile for, nor even how the existing architectures would call C code.
You'd have to invent that as part of the retargeting to the emacs vm.

Doug

----------------------------------------

I believe the most viable way to use SBCL for Emacs would be to rethink
the approach: instead of a graphics engine written in C - in addition to
many Lisp functions - and Elisp used to "script" that, with SBCL you no
longer need to have much of that C code for performance reasons.

The entry point to this hypothetical new Emacs would be in Common Lisp,
as well as an Elisp environment that would be mostly macrology on top of
CL (to start with). After that, you could consider rewriting much of the
rendering engine into Lisp too, and leave only a tiny amount of C
strictly for interfacing with the operating system.

I know some people have discussed about turning SBCL into a shared
library but the amount of work would be considerable and with little
gain because I doubt that the SBCL maintainers are interested in
committing to a stable C ABI to the compiler internals - otherwise
embedding SBCL into a C program would be of little use except for maybe
running CL away from the main thread.

Stelian Ionescu

----------------------------------------

I expect portability to be a real issue here. If you *do* want to rely on S=
BCL to generate native machine code for you, and abandon supporting platfor=
ms that SBCL doesn't, that would be OK. This would entail significantly mod=
ifying the Emacs runtime though. The SBCL compiler is married pretty heavil=
y to its runtime (mainly through the GC, which is written in C), and SBCL i=
s a small C program which simply jumps to the Lisp entry point in a large L=
isp image and stays there in its own world, with its own calling convention=
 and object layour, occasionally calling back out to C for operating system=
 tools and GC, which understands Lisp objects and calling conventions. So L=
isp/C interoperation works very well, but as others have stated, it works i=
n a very SBCL-specific manner. I reckon the hard part would be getting the =
C engine for Emacs to fit in the picture, hence the suggestion to rewrite m=
uch of it in Lisp.
But the reality is I sort of doubt that eliminating support for a wide vari=
ety of other architectures is acceptable to Emacs. Hence the suggestion to =
create an Emacs bytecode backend. This would solve the portability problem,=
 at the cost of some efficiency. SBCL tries much harder than Emacs to produ=
ce good code (by utilizing high level optimizations such as constraint (e.g=
. type) propagation), however, so even then the bytecode produced by SBCL w=
ould be superior compared to what Emacs can produce now. A problem with thi=
s is that the compiler really does expect to be targeting a CPU ISA rather =
than something higher level, so you'll have to be clever to figure out how =
to set up the translation from the machine independent intermediate represe=
ntation (IR2), and Elisp bytecode.
Finally, another option to reap the benefits of SBCL's fastish code generat=
ion while not sacrificing portability is to figure out how to make Emacs a =
portable Common Lisp program, through use of portable libraries for foreign=
 code interop, graphics, and Elisp (through macrology, most likely). Then i=
t will run on all platforms that have C compilers through other Lisp implem=
entations which do not target machine code (like ECL, CLISP, etc...) and us=
e SBCL for the platforms it supports. This approach is taken by StumpWM, wh=
ich is fairly similar to Emacs.
=20
Charles

----------------------------------------

On Tue, May 07, 2019 at 10:22:39AM -0400, Stefan Monnier wrote:
>> embed ECL (Embeddable Common Lisp), which
>> * is significantly slower than SBCL, about 2~3x slower? but is still
>>   much faster than Elisp.
>
>Last time this discussion came up, ECL seemed like the most promising
>option indeed, but the performance was not that impressive compared to
>Emacs.  Maybe the situation has changed?
>Also in terms of maintenance, it's minimal, so it wouldn't help very
>much on the side of manpower.
>
>
>        Stefan
>
>