Re: How to add pseudo vector types

[Yuan Fu <casouri@gmail.com>]

> On Jul 28, 2021, at 12:43 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 28 Jul 2021 12:36:33 -0400
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>> Clément Pit-Claudel <cpitclaudel@gmail.com>,
>> monnier@iro.umontreal.ca, emacs-devel <emacs-devel@gnu.org>
>> 
>>> What, exactly, will the buffer-text fetch code do if tree-sitter
>>> violates the narrowing (by some error in tree-sitter or user code)?
>>> throw an exception? return a null string?
>> 
>> In my current implementation, if tree-sitter access buffer content outside of narrowed region, it reads whitespaces, if it access buffer content outside of the buffer, it reads null string. Neither case throws an error.
> 
> What does TS expect the reader function to return when it hits the
> beginning or end of buffer text?  I think we should behave the same
> when it tries to go beyond the accessible portion.  There should be no
> difference between going beyond the restriction and going beyond EOB.
> 

It expect the read function set *read_bytes to 0 when it reached the end of the buffer. Tree-sitter never “hit the beginning of the buffer text” because it doesn’t read backward. I’m pretty sure tree-sitter expects to always be able to read from BOB.

Could you describe the desired effect on tree-sitter when the buffer is narrowed? If we just deny accessibility of the hidden region from tree-sitter, tree-sitter is still aware of the hidden text, because it has previously parsed the hidden text and stored the result in the parse tree.

My current implementation is to “replace” the hidden region with whitespaces. When the buffer is narrowed and tree-sitter is asked to re-parse (by some user command), I tell tree-sitter that the hidden portion of the buffer has changed, then during the re-parse, tree-sitter will re-scan those parts, and reads whitespaces.

Yuan