bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode

[Yuan Fu <casouri@gmail.com>]

> On Aug 24, 2024, at 1:28 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> Ping!  Any progress with this?
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sat, 10 Aug 2024 22:10:01 -0700
>> Cc: Ergus <spacibba@aol.com>,
>> 72525@debbugs.gnu.org
>> 
>> 
>> 
>>> On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> 
>>>> Date: Thu, 08 Aug 2024 16:45:42 +0200
>>>> From:  Ergus via "Bug reports for GNU Emacs,
>>>> the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>>>> 
>>>> 
>>>> Hi:
>>>> 
>>>> Using this code:
>>>> 
>>>> ```
>>>> int main()
>>>> {
>>>> abort(); /* 1 */
>>>> abort(); /* 1 */
>>>> }
>>>> ```
>>>> 
>>>> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
>>>> 
>>>> When there is a comment at the end of the line, if I do `mark-sexp`
>>>> (C-M-SPC) consecutively I get this selected regions:
>>>> 
>>>> -----------------------------
>>>> 1.
>>>> abort();
>>>> 
>>>> 2.
>>>> abort(); /* 1 */
>>>> 
>>>> 3.
>>>> abort(); /* 1 */
>>>> abort
>>>> 
>>>> 4.
>>>> abort(); /* 1 */
>>>> abort()
>>>> 
>>>> 5.
>>>> abort(); /* 1 */
>>>> abort();
>>>> 
>>>> 6.
>>>> abort(); /* 1 */
>>>> abort(); /* 1 */
>>>> 
>>>> -------------------------------
>>>> 
>>>> 
>>>> 
>>>> But when there is NOT trailing comment
>>>> 
>>>> ```
>>>> int main()
>>>> {
>>>> abort();
>>>> abort();
>>>> }
>>>> ```
>>>> 
>>>> -------------------------------
>>>> 1.
>>>> abort();
>>>> 
>>>> 2.
>>>> abort();
>>>> abort();
>>>> -------------------------------
>>>> 
>>>> 
>>>> It looks like in the fist example after 3 the sexp definition is more fine
>>>> grained (similar to the previous c++-mode behavior) and it selects
>>>> separately:
>>>> the function name,
>>>> the arguments
>>>> the semicolon
>>>> the comment
>>>> 
>>>> But if there is no comment at the end, it always considers the complete
>>>> line as a sexp (including the ;).
>>>> 
>>>> For my use case I would prefer the old behavior because it is consistent
>>>> with the current sexp definition in all emacs (with maybe the exception
>>>> of python-mode).  Because it is easier to copy function names or
>>>> function calls with a few movements.
>>>> 
>>>> However, if it is too difficult to reproduce the old behavior; then the
>>>> new one may be implemented consistently.
>>> 
>>> Yuan, any comments or suggestions?
>>> 
>>> FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
>>> source code is not well-defined.
>> 
>> Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.
>> 
>> Yuan

I know the reason for the inconsistency now. Treesit-forward-sexp first checks whether point is in a “text” node, ie, comment or string; if so, it uses the default/normal forward-sexp function; if not, it uses the parse tree to go over sexp.

In the first example, because there’s a comment right before point, treesit-forward-sexp thinks it’s in a text node, and used the normal forward-sexp function, which moved point after the next symbol.

In the second example, because there’s no comment anymore, treesit-forward-sexp uses the parse tree to move the point; and since the next node after point is the statement line, it moves point over the whole line. When using the parse tree to move point, treesit-forward-sexp always moves in the same “level” in which that point is. Eg, when point is between two lines, treesit-forward-sexp moves over lines; if point is inside an argument list between two arguments, treesit-forward-sexp moves over each argument.

If you want to just select the identifier or other more fine-grained movement, IMHO it’s probably better to use forward-word.

I fixed the inconsistency so now treesit-forward-sexp in both example moves over the whole line. The fix is pushed to emacs-30.

Yuan