unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Re: Using incremental parsing in Emacs (via: emacs rendering comparisson between emacs23 and emacs26.3)
@ 2020-03-30 13:43 Tuấn Anh Nguyễn
  0 siblings, 0 replies; 10+ messages in thread
From: Tuấn Anh Nguyễn @ 2020-03-30 13:43 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: emacs-devel

> The existing third party packages should be good starting points to come
> up with a design.  But I think an important issue is to figure out how
> to make tree-sitter usable for the end users: AFAICT the main issue
> being how to let end users download and install new grammars.
> IIUC grammars are written in Javascript (or some subset thereof?) and
> then somehow compiled to C code.  Having them as C code implies either
> the end-user need to have a C compiler or distributing pre-compiled
> binaries with all the trouble this entails (with all the variations of
> OSes, and architectures, and ABIs, ..., plus issues related to
> licensing, security, ...).

In the short term, I think the practical approach would be distributing
pre-compiled binaries for major targets, and providing users with enough
tooling to compile on other targets. Licensing and security are a
different matter.

> Maybe those grammars could be compiled to some other representation (I
> don't know if it is made mostly of data-tables or actual code or what)?

Many of those grammars have parts of the lexer in custom C code.
Therefore, a common representation would require an instruction set. In
principle, that can be done with something like WebAssembly. Tree-sitter
already compiles to WebAssembly (both the runtime and the grammars). Its
playground uses that:
https://tree-sitter.github.io/tree-sitter/playground.

--
Tuấn-Anh Nguyễn
Software Engineer



^ permalink raw reply	[flat|nested] 10+ messages in thread
* Re: Using incremental parsing in Emacs (via: emacs rendering comparisson between emacs23 and emacs26.3)
@ 2020-03-29 18:46 Stefan Monnier
  2020-03-29 19:05 ` Andrea Corallo
  0 siblings, 1 reply; 10+ messages in thread
From: Stefan Monnier @ 2020-03-29 18:46 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

>> tree-sitter, like LSP, is something Emacs should embrace.
>   https://lists.gnu.org/archive/html/emacs-devel/2020-01/msg00059.html

Ah, thanks Eli: I guess I skipped over that while catching up.

> Would someone like to try to figure out how we could use the
> incremental parsing technology in Emacs for making our
> programming-language support more accurate and efficient?  One package
> that implements this technology is tree-sitter:
>
>   https://tree-sitter.github.io/tree-sitter/

Yes, adding support for this would be great.  

> AFAIU, these capabilities could be used as an alternative to
> regexp- and syntax-pps-based font-lock, better code folding,
> completion, refactoring, and other similar features; in general, any
> feature which would benefit from having a parse tree for the source
> code in a buffer.

Some of those features could be provided by LSP as well, but IIUC the
way LSP is designed and usually used makes it somewhat inadequate for
synchronous use, when you want an immediate answer.

tree-sitter is designed exactly for that: it can parse "immediately",
in the same sense as `syntax-ppss`, so LSP seems inapplicable (in the
near future at least) for things like font-lock and navigation, and
indentation, whereas tree-sitter should work great for that.

[ W.r.t disucssions around LSP's use of JSON: AFAICT, parsing and
  emitting json can be done as efficiently as any other format, AFAICT,
  so I don't see the use of JSON as a problem in the protocol.  ]

> To be able to use such libraries, we need to figure out how to
> integrate them into the core, what kind of interfaces would be needed
> for that, and what kind of infrastructure we would need for basing
> Lisp features on those libraries.

The existing third party packages should be good starting points to come
up with a design.  But I think an important issue is to figure out how
to make tree-sitter usable for the end users: AFAICT the main issue
being how to let end users download and install new grammars.
IIUC grammars are written in Javascript (or some subset thereof?) and
then somehow compiled to C code.  Having them as C code implies either
the end-user need to have a C compiler or distributing pre-compiled
binaries with all the trouble this entails (with all the variations of
OSes, and architectures, and ABIs, ..., plus issues related to
licensing, security, ...).

Maybe those grammars could be compiled to some other representation (I
don't know if it is made mostly of data-tables or actual code or what)?


        Stefan




^ permalink raw reply	[flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-04-01  0:39 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-03-30 13:43 Using incremental parsing in Emacs (via: emacs rendering comparisson between emacs23 and emacs26.3) Tuấn Anh Nguyễn
  -- strict thread matches above, loose matches on Subject: below --
2020-03-29 18:46 Stefan Monnier
2020-03-29 19:05 ` Andrea Corallo
2020-03-29 19:18   ` Eli Zaretskii
2020-03-30  3:35     ` Stefan Monnier
2020-03-30  6:02       ` Eli Zaretskii
2020-03-30 13:33         ` Stefan Monnier
2020-03-30 14:09           ` Eli Zaretskii
2020-03-30 15:03             ` Stefan Monnier
2020-04-01  0:39               ` Stephen Leake

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).