Re: Emacs Lisp JIT Compiler

["Daniel Colascione" <dancol@dancol.org>]

>> From: Tom Tromey <tom@tromey.com>
>> Cc: Paul Eggert <eggert@cs.ucla.edu>,  dancol@dancol.org,
>> tom@tromey.com,  emacs-devel@gnu.org
>> Date: Wed, 15 Aug 2018 18:29:37 -0600
>>
>> That said I don't think it is a very important issue, I think primarily
>> because interested users can port the JIT as needed.
>
> Only if they are _very_ interested, and have the knowledge/talent to
> do that.

It only takes one such person though.

>> Eli> That doesn't mean we want to have predictably bad performance, but
>> it
>> Eli> does mean that we should choose our JIT library so that it doesn't
>> Eli> suffer from such issues.
>>
>> There is no JIT library that supports everything.
>
> OK, let me rephrase: we should choose our JIT library so that it
> suffers as little as possible from such issues.

Right. And I think a standalone library is less likely to suffer from
these issues than one that requires a full toolchain. It's easier to port
back-end machine-code generation to an architecture than it is to port all
of binutils.

I also strongly suspect (albeit without numbers ATM) that the standalone
approach will yield much better performance than one which involves a trip
through the filesystem and a compilation toolchain. A standalone system
lets us do tiered and profile-guided compilation in a way that a big batch
process really can't accommodate due to overheads.

I think it's also really important to emphasize the "IT" aspect of the JIT
system. That is, there are tons of cases where it would be a huge help to
generate specialized code (e.g., for parsing) and compile it on-the-fly.
Imagine we have decent elisp parsers for languages A and B, but we (in
some distant and glittering future) have robust multi-mode support and
want to parse the language (A|B) in the buffer. Wouldn't it be nice to
just synthesize A|B into some code and compile that straight to machine
code?

Likewise, it'd be fantastic to compile regular expressions to DFAs and
then generate machine code for the DFAs. You can't go faster than that.

All of these use cases need a level of dynamism that I'm afraid we
wouldn't be able to get with good performance via a toolchain-based
system.