bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Yuan Fu]

X-Debbugs-CC: theo@thornhill.no

In tsx-ts-mode we use this query to apply syntax properties:


(defvar tsx-ts--s-p-query
  (when (treesit-available-p)
    (treesit-query-compile 'tsx
                           '(((regex pattern: (regex_pattern) @regexp))
                             ((variable_declarator value: (jsx_element) @jsx))
                             ((assignment_expression right: (jsx_element) @jsx))
                             ((arguments (jsx_element) @jsx))
                             ((parenthesized_expression (jsx_element) @jsx))
                             ((return_statement (jsx_element) @jsx))))))


And then in tsx-ts--syntax-propertize-captures we mark everything
enclosed by the captured jsx_element nodes in text fences.

Then for the following code

<button onClick={() => {
  func();
  return true;
}}>
  Text
  {func();}
</button>

All the func() and other code will be considered text because the whole
jsx tag (<button>...</button>) are wrapped in string fences. Theo,
what’s the original intention for marking jsx_elements as text? Can we
only mark jsx_text as string?

Yuan

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Eli Zaretskii]

Ping! Theo, can you answer Yuan's questions?

> Cc: theo@thornhill.no
> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 23 Oct 2024 21:06:40 -0700
> 
> X-Debbugs-CC: theo@thornhill.no
> 
> In tsx-ts-mode we use this query to apply syntax properties:
> 
> 
> (defvar tsx-ts--s-p-query
>   (when (treesit-available-p)
>     (treesit-query-compile 'tsx
>                            '(((regex pattern: (regex_pattern) @regexp))
>                              ((variable_declarator value: (jsx_element) @jsx))
>                              ((assignment_expression right: (jsx_element) @jsx))
>                              ((arguments (jsx_element) @jsx))
>                              ((parenthesized_expression (jsx_element) @jsx))
>                              ((return_statement (jsx_element) @jsx))))))
> 
> 
> And then in tsx-ts--syntax-propertize-captures we mark everything
> enclosed by the captured jsx_element nodes in text fences.
> 
> Then for the following code
> 
> <button onClick={() => {
>   func();
>   return true;
> }}>
>   Text
>   {func();}
> </button>
> 
> All the func() and other code will be considered text because the whole
> jsx tag (<button>...</button>) are wrapped in string fences. Theo,
> what’s the original intention for marking jsx_elements as text? Can we
> only mark jsx_text as string?
> 
> Yuan
> 
> 
> 
> 
> 

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Yuan Fu]

> On Nov 9, 2024, at 1:11 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> Ping! Theo, can you answer Yuan's questions?

I’ve been using my local fix at work for a while now and it seems to work fine. I’ll make a patch and apply in a few days.

Yuan

> 
>> Cc: theo@thornhill.no
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 23 Oct 2024 21:06:40 -0700
>> 
>> X-Debbugs-CC: theo@thornhill.no
>> 
>> In tsx-ts-mode we use this query to apply syntax properties:
>> 
>> 
>> (defvar tsx-ts--s-p-query
>>  (when (treesit-available-p)
>>    (treesit-query-compile 'tsx
>>                           '(((regex pattern: (regex_pattern) @regexp))
>>                             ((variable_declarator value: (jsx_element) @jsx))
>>                             ((assignment_expression right: (jsx_element) @jsx))
>>                             ((arguments (jsx_element) @jsx))
>>                             ((parenthesized_expression (jsx_element) @jsx))
>>                             ((return_statement (jsx_element) @jsx))))))
>> 
>> 
>> And then in tsx-ts--syntax-propertize-captures we mark everything
>> enclosed by the captured jsx_element nodes in text fences.
>> 
>> Then for the following code
>> 
>> <button onClick={() => {
>>  func();
>>  return true;
>> }}>
>>  Text
>>  {func();}
>> </button>
>> 
>> All the func() and other code will be considered text because the whole
>> jsx tag (<button>...</button>) are wrapped in string fences. Theo,
>> what’s the original intention for marking jsx_elements as text? Can we
>> only mark jsx_text as string?
>> 
>> Yuan
>> 
>> 
>> 
>> 
>> 

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 9 Nov 2024 08:49:55 -0800
> Cc: Theodor Thornhill <theo@thornhill.no>,
>  73978@debbugs.gnu.org
> 
> 
> 
> > On Nov 9, 2024, at 1:11 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> > Ping! Theo, can you answer Yuan's questions?
> 
> I’ve been using my local fix at work for a while now and it seems to work fine. I’ll make a patch and apply in a few days.

Did you have an opportunity to install such a patch?

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Yuan Fu]

> On Nov 23, 2024, at 4:15 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 9 Nov 2024 08:49:55 -0800
>> Cc: Theodor Thornhill <theo@thornhill.no>,
>> 73978@debbugs.gnu.org
>> 
>> 
>> 
>>> On Nov 9, 2024, at 1:11 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>> Ping! Theo, can you answer Yuan's questions?
>> 
>> I’ve been using my local fix at work for a while now and it seems to work fine. I’ll make a patch and apply in a few days.
> 
> Did you have an opportunity to install such a patch?

Hey sorry, I haven’t applied the patch. Actually, I want to ask you a question before I do: is there a way to mark a single character in buffer in string syntax? The only way I’m aware of is to mark string delimiter syntax to the start and end of the string, but that doesn’t work for a single character.

Take the following snippet as an example:

<button>a<button>

I want to apply string syntax to “a”.

If there’s no such way, I guess just not applying the string syntax in such case is also an option.

Yuan

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Sat, 23 Nov 2024 21:25:50 -0800
> Cc: Theodor Thornhill <theo@thornhill.no>,
>  73978@debbugs.gnu.org
> 
> >> I’ve been using my local fix at work for a while now and it seems to work fine. I’ll make a patch and apply in a few days.
> > 
> > Did you have an opportunity to install such a patch?
> 
> Hey sorry, I haven’t applied the patch. Actually, I want to ask you a question before I do: is there a way to mark a single character in buffer in string syntax? The only way I’m aware of is to mark string delimiter syntax to the start and end of the string, but that doesn’t work for a single character.
> 
> Take the following snippet as an example:
> 
> <button>a<button>
> 
> I want to apply string syntax to “a”.
> 
> If there’s no such way, I guess just not applying the string syntax in such case is also an option.

There's a syntax-table text property, see the node "Syntax Properties"
in the ELisp manual.  Would that do the job?

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Dmitry Gutov]

On 24/11/2024 09:47, Eli Zaretskii wrote:
>> Hey sorry, I haven’t applied the patch. Actually, I want to ask you a question before I do: is there a way to mark a single character in buffer in string syntax? The only way I’m aware of is to mark string delimiter syntax to the start and end of the string, but that doesn’t work for a single character.
>>
>> Take the following snippet as an example:
>>
>> <button>a<button>
>>
>> I want to apply string syntax to “a”.
>>
>> If there’s no such way, I guess just not applying the string syntax in such case is also an option.
> There's a syntax-table text property, see the node "Syntax Properties"
> in the ELisp manual.  Would that do the job?

In particular, the "generic string" syntax property, this one

   (string-to-syntax "|")

You put it on the first and the last chars of a "generic string".

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Yuan Fu]

> On Nov 24, 2024, at 5:45 AM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
> On 24/11/2024 09:47, Eli Zaretskii wrote:
>>> Hey sorry, I haven’t applied the patch. Actually, I want to ask you a question before I do: is there a way to mark a single character in buffer in string syntax? The only way I’m aware of is to mark string delimiter syntax to the start and end of the string, but that doesn’t work for a single character.
>>> 
>>> Take the following snippet as an example:
>>> 
>>> <button>a<button>
>>> 
>>> I want to apply string syntax to “a”.
>>> 
>>> If there’s no such way, I guess just not applying the string syntax in such case is also an option.
>> There's a syntax-table text property, see the node "Syntax Properties"
>> in the ELisp manual.  Would that do the job?
> 
> In particular, the "generic string" syntax property, this one
> 
>  (string-to-syntax "|")
> 
> You put it on the first and the last chars of a "generic string”.

The problem is, that doesn’t work when there’s only one character. Take the snippet as an example:

<button>a</button>

You can’t put the string fence syntax on the “a”, because there isn’t a closing fence to close it.

Yuan

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Yuan Fu]

> On Nov 24, 2024, at 5:27 PM, Yuan Fu <casouri@gmail.com> wrote:
> 
> 
> 
>> On Nov 24, 2024, at 5:45 AM, Dmitry Gutov <dmitry@gutov.dev> wrote:
>> 
>> On 24/11/2024 09:47, Eli Zaretskii wrote:
>>>> Hey sorry, I haven’t applied the patch. Actually, I want to ask you a question before I do: is there a way to mark a single character in buffer in string syntax? The only way I’m aware of is to mark string delimiter syntax to the start and end of the string, but that doesn’t work for a single character.
>>>> 
>>>> Take the following snippet as an example:
>>>> 
>>>> <button>a<button>
>>>> 
>>>> I want to apply string syntax to “a”.
>>>> 
>>>> If there’s no such way, I guess just not applying the string syntax in such case is also an option.
>>> There's a syntax-table text property, see the node "Syntax Properties"
>>> in the ELisp manual.  Would that do the job?
>> 
>> In particular, the "generic string" syntax property, this one
>> 
>> (string-to-syntax "|")
>> 
>> You put it on the first and the last chars of a "generic string”.
> 
> The problem is, that doesn’t work when there’s only one character. Take the snippet as an example:
> 
> <button>a</button>
> 
> You can’t put the string fence syntax on the “a”, because there isn’t a closing fence to close it.
> 
> Yuan

Circling back on this. I don’t think there’s a way to apply string syntax to a single character.

Yuan

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Dmitry Gutov]

On 11/12/2024 06:52, Yuan Fu wrote:
>> The problem is, that doesn’t work when there’s only one character. Take the snippet as an example:
>>
>> <button>a</button>
>>
>> You can’t put the string fence syntax on the “a”, because there isn’t a closing fence to close it.
>>
>> Yuan
> Circling back on this. I don’t think there’s a way to apply string syntax to a single character.

Indeed, sorry.

There needs to be a separate char as a "closing fence" like you say 
because it's treated as a part of the string. So it's 2 chars minimum.

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Yuan Fu]

> On Dec 11, 2024, at 6:52 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
> On 11/12/2024 06:52, Yuan Fu wrote:
>>> The problem is, that doesn’t work when there’s only one character. Take the snippet as an example:
>>> 
>>> <button>a</button>
>>> 
>>> You can’t put the string fence syntax on the “a”, because there isn’t a closing fence to close it.
>>> 
>>> Yuan
>> Circling back on this. I don’t think there’s a way to apply string syntax to a single character.
> 
> Indeed, sorry.
> 
> There needs to be a separate char as a "closing fence" like you say because it's treated as a part of the string. So it's 2 chars minimum.

How hard is it to add a new syntax for this case? Or is there some way to work around this? We can’t just not apply the string syntax, because if the “a” is a parenthesis, etc, it would mess up the parenthesis balancing after it.

Maybe just give it a whitespace syntax? 

Yuan

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Dmitry Gutov]

On 12/12/2024 06:56, Yuan Fu wrote:
> How hard is it to add a new syntax for this case? Or is there some way to work around this? We can’t just not apply the string syntax, because if the “a” is a parenthesis, etc, it would mess up the parenthesis balancing after it.

Probably not very hard, but that seems like it'd affect the total set of 
syntax classes - which means adding it to the manual, etc.

> Maybe just give it a whitespace syntax?

Right, in such cases I applied the "whitespace" or "punctuation" syntax 
to the whole character span, like in 
https://github.com/dgutov/mmm-mode/blob/master/mmm-erb.el#L97

bug#73978: 31.0.50; Text syntax applied on too many things in tsx-ts-mode

[Yuan Fu]

> On Dec 12, 2024, at 9:19 AM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
> On 12/12/2024 06:56, Yuan Fu wrote:
>> How hard is it to add a new syntax for this case? Or is there some way to work around this? We can’t just not apply the string syntax, because if the “a” is a parenthesis, etc, it would mess up the parenthesis balancing after it.
> 
> Probably not very hard, but that seems like it'd affect the total set of syntax classes - which means adding it to the manual, etc.
> 
>> Maybe just give it a whitespace syntax?
> 
> Right, in such cases I applied the "whitespace" or "punctuation" syntax to the whole character span, like in https://github.com/dgutov/mmm-mode/blob/master/mmm-erb.el#L97

Thanks. I went with the whitespace trick and pushed my patch to master.

Yuan