bug#62333: 30.0.50; Issue with tree-sitter syntax tree during certain changes

[Dmitry Gutov <dgutov@yandex.ru>]

On 27/03/2023 16:29, Eli Zaretskii wrote:
>> Date: Mon, 27 Mar 2023 01:00:20 +0300
>> Cc: wkirschbaum@gmail.com, casouri@gmail.com, 62333@debbugs.gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>>> So you are suggesting to introduce kludges like font-lock-dont-widen
>>> in all of those places?
>>
>> font-lock-dont-widen is a kludge, but that's largely determined by the
>> way it's defined and used.
> 
> With parser-based features, we have an opportunity to do this in a
> cleaner manner.

parser-based features don't need this at all, if considered in 
isolation. But if we try to combine them with existing mode, or existing 
packages, they need to play by the common rules. Which they currently do.

>> If we take indent-for-tab-command, for example, it doesn't have such a
>> variable, and doesn't really need to: the top-level command calls
>> 'widen', and then indent-line-function (set by major mode and altered by
>> e.g. mmm-mode) is free to impose its specific bounds.
> 
> I thought about a lower-level, infrastructure-level, mechanism that
> could be used to restrict a parser to a certain region of the buffer.
> Then this could be used by every feature based on parsers, instead of
> us having to invent a separate solution for each one.

Like narrowing, but just for parsers? But parsers obey narrowing 
already. Sounds a bit like conceptual duplication. How does this solve 
blink-matching-paren issue anyway?

>> The "grand unified theory of mixed major modes" has been attempted a few
>> times in the past, and never reached anything practical.
> 
> But here we have a unique opportunity to maybe find a solution, at
> least for stuff based on tree-sitter and similar libraries.  That
> maybe not "grand", but certainly "respectable".

tree-sitter has its own support for mixed languages.

>>>> Except it's already limited by narrowing.
>>>
>>> Which is a fragile, semi-broken means, as we all know.
>>
>> What is a broken mess, is user-level narrowing.
> 
> Which is why restricting parsers should not be a user-level feature.

We're talking elisp-level, though. The problem with interactive 
narrowing, is its impossible to tell apart from one effected by Lisp. 
And they usually have different goals.

>>> Look, I'm trying to see how we could come up with an infrastructure
>>> that will support multiple modes and other similar features in the
>>> same buffer without relying on narrowing, thus bypassing the
>>> disadvantages and difficulties that come with narrowing.  I think we
>>> have a good chance here to come up with such a solution, specifically
>>> for features that us a parsing library.  If you aren't interested in
>>> discussing that, and think we should stick to narrowing, then this
>>> goes nowhere, and I'd rather bow out of it.
>>
>> What I've seen here so far is you suggesting we go ahead and break the
>> existing convention and then let "them" (third-party authors including
>> myself) come up with a new working one.
> 
> You are mixing two semi-separate issues: how to fix the immediate
> problem with blink-matching-paren (if we consider it a serious
> problem), and how to try solving similar issues as a whole.

Not mixing anything. We have one proposal for improving 
blink-matching-paren integration anyway. It should be compatible with 
whatever we choose regarding narrowing and mixed modes.

> The
> latter is a long-term goal, and we can discuss it freely and calmly;
> whereby ideas can be brought up even if they might not be the most
> clever ones or the ones with which you personally agree.

I'm not looking for "clever".

>> My stance here is we shouldn't break it before we create a new one.
> 
> No one broke anything.  We are just discussing ideas.  Please don't
> exaggerate.

I never said anybody has broken anything already.