Re: Using incremental parsing in Emacs

[Eli Zaretskii <eliz@gnu.org>]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Fri, 03 Jan 2020 15:53:45 -0800
> 
> The interface should look like LSP; it aims to support everything an IDE
> needs from a "language server" (ie parser), and allows for custom
> extensions where it falls short.

Maybe I'm the odd one out, but I don't think I have a clear idea of
what the "LSP interface" entails.  Would you (or someone else) post a
summary, or point to some place where this is described succinctly
enough to not require a long study?

We did learn one important thing from using LSP servers: that
processing the JSON stuff back and forth adds non-trivial overhead and
slows down the application enough to annoy users, even after we did
all we can to speed up the translation.  So I think it makes sense to
take one more look at the issue and see if we can come up with better
interfaces, which will suit Emacs applications better and allow faster
processing.  Using a library that processes stuff locally would then
allow us to implement such interfaces more easily, since we will be
free from the restrictions imposed by the need to communicate with
external processes.

we'll most probably want some combination of LSP-based and local
parsers-based features.  E.g., it's quite possible that LSP servers
could be better for some complex jobs, where speed matters less.

My point is that we shouldn't lock up our minds, not yet anyway.  A
fresh look at these issues, taking the incremental parsing into
account, could benefit us in the long run.

> LSP language servers are implemented in some compiled language, not
> elisp; eglot/lsp-mode is just the elisp side of the protocol. The elisp
> sends edits and info requests (ie, "insert/delete this text at this
> point", "fontify/format this range") to the server, and handles the
> responses.

I'm saying we should look into this and see whether there are better
ways that that.  Suppose such a server had direct access to buffer
text: would that allow a more efficient interface than the above?  We
shouldn't be "dragged" after the LSP ideas just because they are
there, and we shouldn't automatically decide that an interface is OK
just because it can be done in Lisp with no changes on the C level.
Emacs has some unique features and requirements that might affect the
design of the interfaces to such servers and libraries.

We've been there several times with minor features like line numbers
and fill-column indicator.  Why would this issue be an exception?
Especially since we have lately quite a "schism" among our active
developers: those who have the most experience in implementing such
applications aren't familiar well enough with the C internals, and
avoid making changes there.  This could very well be an obstacle to
coming up with good ideas regarding the best interfaces and
implementation options for the related features.

> ada-mode works in a similar way, but LSP is an industry standard, so it
> is a better choice.
> 
> If Emacs has a mode that conforms to the editor side of the LSP
> protocol, it can use _any_ LSP language server; they do not have to be
> provided by Emacs.

We should definitely support LSP.  We already do, albeit in
third-party packages.  We added native JSON support and jsonrpc for
doing this better.  If there's anything else we can do in that
direction, people should speak up.

But my point is that LSP is not necessarily the only game in town we
should support.  For example, font-lock doesn't use LSP, and probably
never will, due to performance issues; should we improve font-lock
using infrastructure that's based on language parsing?  And there are
other features that could benefit, I've mentioned them.  If you are
saying they all should just use LSP, then I don't think I agree.