bug#59693: 29.0.50; treesitter in base buffer doesn't respond to modifications in indirect buffer correctly

[Yuan Fu <casouri@gmail.com>]

> On Dec 3, 2022, at 11:46 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 3 Dec 2022 23:20:59 -0800
>> Cc: miha@kamnitnik.top,
>> 59693@debbugs.gnu.org
>> 
>>>>> We don’t want indirect buffer and base buffers to share parsers, since they can have different narrowing, and semantically indirect buffers should share anything but the text with the base buffer.
>>>> 
>>>> Yes, the parsers should not be shared.
>>>> 
>>>>> How about this: we change current_buffer->parser_list from a plain list of parsers to a cons (PARSER-LIST . INDIRECT-PARSER-LIST), where PARSER-LIST is as before. But for base buffers, INDIRECT-PARSER-LIST includes all the parsers of its indirect buffers; and for indirect buffers, INDIRECT-PARSER-LIST is nil.
>>>> 
>>>> You can maybe have the indirect buffers in the list, not their parsers.
>>>> That could make it easier to access other treesit-related information of the
>>>> indirect buffers, if needed.
>>> 
>>> Good idea, it’s easier to know when to remove the reference with buffers, aka when buffer is killed.
>> 
>> I now have a patch that fixes this problem. WDYT? I added a new buffer field since it’s cleaner than turning ts_parser_list into a cons, hopefully that’s not frowned upon. 
> 
> Thanks.
> 
> If we are adding to the buffer object a field that holds the list of
> indirect buffers, then:
> 
>  . the name of the field should not include "treesit" in it, and it
>    shouldn't be conditioned on HAVE_TREE_SITTER

I made it tree-sitter specific and stated that it doesn’t necessarily contain all indirect buffers to simply the implementation. Right now new indirect buffer are added to the list only when a parser is created in an indirect buffer. And dead indirect buffers are discarded only when treesit_record_change runs. Do we really need a proper indirect buffer list? After all, no one complained about its absence ever since indirect buffer were added.

>  . I wonder if a flat list is a good idea, i.e. scalable enough? also,
>    treesit_reap_indirect_buffers conses a lot as result

AFAIK in most cases only a handful of indirect buffer are created, so I didn’t worry about that. For tree-sitter’s case, we need to iterate through every indirect buffer anyway so that’s always O(n). Adding and removing a buffer from the list is O(n) but they are done infrequently. The add operation is called every time a parser is created, and the remove operation is called once when an indirect buffer is killed. 

We could use a hash table, but I doubt if that’s necessary, at least while the indirect buffer list is only used for tree-sitter. For tree-sitter, the common path is when we update each parser of buffer changes, which is always O(n) regardless of the data structure. 

>  . I vaguely remember that adding built-in fields to the buffer object had
>    some caveats, but I don't recall the details (did you bootstrap?)

I built from git clean, so yeah.

> 
> Stefan, any comments on this?  Are there better ideas of how to track buffer
> text changes in indirect buffers?

Specifically, when any one of the base and indirect buffers change, we want all parsers in all base and indirect buffers to be updated.

Yuan