Re: How to add pseudo vector types

[Yuan Fu <casouri@gmail.com>]

> On Jul 16, 2021, at 10:23 PM, Clément Pit-Claudel <cpitclaudel@gmail.com> wrote:
> 
> On 7/16/21 10:05 PM, Yuan Fu wrote:
>> My conclusion is that after-change-hook is pretty insignificant, and the initial parse is a bit slow (on large files).
> 
> I have no idea if it makes sense, but: does the initial parse need to be synchronous, or could you instead run the parsing in one thread, and the rest of Emacs in another? (I'm talking about concurrent execution, not cooperative threading).
> 
> In most cases there should be very limited contention, if at at all: in large buffers most of Emacs' activity will be focused on the (relatively few) characters around the gap, and most of the parser's activity will be reading from the buffer at other positions.  You do need to be careful to not read the garbage data from the gap, but otherwise seeing stale or even inconsistent data from the parser thread shouldn't be an issue, since tree-sitter is supposed to be robust to bad parses.
> 
> In fact, depending on how robust tree-sitter is, you might even be able to do the concurrency-control optimistically (parse everything up to close to the gap, check that the gap hasn't moved into the region that you read, and then resume reading or rollback).
> 
> Alternatively, maybe you could even do a full parse with minimal concurrency control: you'd make sure that the Emacs thread records not just changes to the buffer text but also movements of the gap, and then you could use that list of changes for the next parse?

Another way I thought about is to only “expose” the portion of buffer from BOB to some point to tree-sitter. And when a user asks for a parse tree, he also specifies to which point of the buffer he needs the parse tree for. For example, for fortification, jit-lock only needs the tree up to the end of the visible window. And for structure editing, asking for the portion up to window-end + a few thousand characters might be enough. However this heuristic could have problems in practice. (Maybe a giant comment section of thousands of characters follows, and instead of jumping to the end of it, we wrongly jump to middle of that comment section, because tree-sitter only “sees” to that point.) So I don’t know if it’s a good idea.

> 
> Anyway, thanks for working on this!
> 

I figure that this is low-tech enough that an amateur like me could possibly do it ;-)

Yuan