Re: How to add pseudo vector types

["Clément Pit-Claudel" <cpitclaudel@gmail.com>]

Thanks for the detailed reply.

On 7/17/21 3:16 AM, Eli Zaretskii wrote:
> When Emacs moves or enlarges/shrinks the gap, that affects the entire
> buffer text after the gap, regardless of where the gap is.  So it will
> affect the TS reader if it reads stuff after the gap.

Doesn't enlarging the gap require allocating a new buffer and copying data to it?  If so it wouldn't affect the TS reader.  Moving is indeed trickier, that's what I referred to as "limited contention".

>> You do need to be careful to not read the garbage data from the gap, but otherwise seeing stale or even inconsistent data from the parser thread shouldn't be an issue, since tree-sitter is supposed to be robust to bad parses.
> 
> What would be the purpose of calling the parser if we know in advance
> it will fail when it gets to the "garbage" caused by async access to
> the buffer text?

It won't fail, will it?  I thought this was the point of TS, that it would reuse the initial parse on the "good" parts in subsequent parses.

> So I don't see how this could be done without some inter-locking.

Yes, there probably need to be some care around the gap area.  But that's what I was referring to re. "optimistic concurrency".

> And what do you want the code which requested parsing do while the
> parse thread runs?  The requesting code is in the main thread, so if
> it just waits, you don't gain anything.

You'd have the parser running continuously in the background, every time there is a change.  When a piece of code requests a parse it blocks and waits, but presumably for not too long because a very recent previous parse means that the blocking parse is fast.

>> In fact, depending on how robust tree-sitter is, you might even be able to do the concurrency-control optimistically (parse everything up to close to the gap, check that the gap hasn't moved into the region that you read, and then resume reading or rollback).
> 
> I don't understand what you suggest here.  For starters, the gap could
> move (assuming you are still talking about a separate thread that does
> the parsing), and what do we do then?

Nothing, we start the next parse when this one completes.

> I don't understand what could recording the gap solve.  The stuff in
> the gap is generally garbage, and can easily include invalid multibyte
> sequences.  I don't think it's a good idea to pass that to TS.  Also,
> recording the gap changes in the main thread and accessing that
> information from a concurrent thread again opens a window for races,
> and requires synchronization.

This list of gap changes wouldn't be accessed concurrently: you would (message-)pass a copy of it to the parser thread every time it starts a new parse.

> Bottom line, I think what you are suggesting is premature
> optimization: we don't yet know that we will need this. 

I thought we knew that a full parse of some files could take over a second; but yes, it will be nice if we can find a synchronous way to avoid having to do a full parse.