bug#71086: 30.0.50; Setting treesit-indent-function is not effective is treesit-simple-indent-rules is nil

bug#71086: 30.0.50; Setting treesit-indent-function is not effective is treesit-simple-indent-rules is nil

[Óscar Fuentes]

treesit-major-mode-setup setups indentation if and only if
treesit-simple-indent-rules is non-nil. So the major mode author is
forced to set that variable even when he makes no use of the
treesit-simple-indent facilities and instead sets
treesit-indent-function to his own function.

Having to assign a dummy value to a variable that belongs to an API that
is not used is surprising. Furthermore, it is unspecified what kind of
values are acceptable on that scenario.

I would expect that treesit-major-mode-setup shall setup indentation
when treesit-indent-function is non-nil.

OTOH, the info node for "Parser-based Indentation" says:

     To enable the parser-based indentation engine, either set
  ‘treesit-simple-indent-rules’ and call ‘treesit-major-mode-setup’, or
  equivalently, set the value of ‘indent-line-function’ to
  ‘treesit-indent’.

However, treesit-major-mode-setup does more than what's described after
"equivalently":

  (when treesit-simple-indent-rules
    (setq-local treesit-simple-indent-rules
                (treesit--indent-rules-optimize
                 treesit-simple-indent-rules))
    (setq-local indent-line-function #'treesit-indent)
    (setq-local indent-region-function #'treesit-indent-region))



In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.18.0) of 2024-05-19 built on sky
Repository revision: 98b83bdc9f4af7798e3314ad8df6ab78efd60f8a
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Debian GNU/Linux trixie/sid

Configured using:
 'configure --with-native-compilation --without-toolkit-scroll-bars
 --with-x-toolkit=lucid --with-modules --without-imagemagick'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBOTF
LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND THREADS TIFF TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2
XPM LUCID ZLIB

Important settings:
  value of $LANG: C
  locale-coding-system: nil

bug#71086: 30.0.50; Setting treesit-indent-function is not effective is treesit-simple-indent-rules is nil

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 21 May 2024 03:32:05 +0200
> 
> 
> treesit-major-mode-setup setups indentation if and only if
> treesit-simple-indent-rules is non-nil. So the major mode author is
> forced to set that variable even when he makes no use of the
> treesit-simple-indent facilities and instead sets
> treesit-indent-function to his own function.

If your mode defines a treesit-indent-function, why do you need the
help of treesit-major-mode-setup for setting up indentation?

The idea of treesit-simple-indent, AFAIU, is to automate the simple
cases, on the assumption that the mode complex cases cannot be
second-guessed up front, and the mode programmer will need to set that
up manually.  For example, you can assign your functions to
indent-line-function and indent-region-function.  Or what am I
missing?

> OTOH, the info node for "Parser-based Indentation" says:
> 
>      To enable the parser-based indentation engine, either set
>   ‘treesit-simple-indent-rules’ and call ‘treesit-major-mode-setup’, or
>   equivalently, set the value of ‘indent-line-function’ to
>   ‘treesit-indent’.
> 
> However, treesit-major-mode-setup does more than what's described after
> "equivalently":
> 
>   (when treesit-simple-indent-rules
>     (setq-local treesit-simple-indent-rules
>                 (treesit--indent-rules-optimize
>                  treesit-simple-indent-rules))
>     (setq-local indent-line-function #'treesit-indent)
>     (setq-local indent-region-function #'treesit-indent-region))

So this is just a matter of updating the documentation to mention
indent-region-function?

I've added Yuan, in case he has comments or suggestions.

bug#71086: 30.0.50; Setting treesit-indent-function is not effective is treesit-simple-indent-rules is nil

[Yuan Fu]

> On May 21, 2024, at 4:43 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 21 May 2024 03:32:05 +0200
>> 
>> 
>> treesit-major-mode-setup setups indentation if and only if
>> treesit-simple-indent-rules is non-nil. So the major mode author is
>> forced to set that variable even when he makes no use of the
>> treesit-simple-indent facilities and instead sets
>> treesit-indent-function to his own function.
> 
> If your mode defines a treesit-indent-function, why do you need the
> help of treesit-major-mode-setup for setting up indentation?
> 
> The idea of treesit-simple-indent, AFAIU, is to automate the simple
> cases, on the assumption that the mode complex cases cannot be
> second-guessed up front, and the mode programmer will need to set that
> up manually.  For example, you can assign your functions to
> indent-line-function and indent-region-function.  Or what am I
> missing?

There are actually another level of customization between setting treesit-simple-indent-rules and setting indent-line-function, which is setting treesit-indent-function. IIUC Óscar is trying to do that but found out treesit-major-mode-setup doesn’t automatically setup indent-line-functions when treesit-indent-function is set. I’ve now fixed that. (Though I didn’t expect someone to use this variable.)

And to Óscar, I’m curious and interested to know what indentation function you came up with; if it’s useful maybe we can add it to Emacs?

> 
>> OTOH, the info node for "Parser-based Indentation" says:
>> 
>>     To enable the parser-based indentation engine, either set
>>  ‘treesit-simple-indent-rules’ and call ‘treesit-major-mode-setup’, or
>>  equivalently, set the value of ‘indent-line-function’ to
>>  ‘treesit-indent’.
>> 
>> However, treesit-major-mode-setup does more than what's described after
>> "equivalently":
>> 
>>  (when treesit-simple-indent-rules
>>    (setq-local treesit-simple-indent-rules
>>                (treesit--indent-rules-optimize
>>                 treesit-simple-indent-rules))
>>    (setq-local indent-line-function #'treesit-indent)
>>    (setq-local indent-region-function #'treesit-indent-region))
> 
> So this is just a matter of updating the documentation to mention
> indent-region-function?
> 
> I've added Yuan, in case he has comments or suggestions.

I also updated the manual.

Yuan

bug#71086: 30.0.50; Setting treesit-indent-function is not effective is treesit-simple-indent-rules is nil

[Óscar Fuentes]

Yuan Fu <casouri@gmail.com> writes:

> There are actually another level of customization between setting
> treesit-simple-indent-rules and setting indent-line-function, which is
> setting treesit-indent-function. IIUC Óscar is trying to do that but
> found out treesit-major-mode-setup doesn’t automatically setup
> indent-line-functions when treesit-indent-function is set.

More specifically, I found that treesit-major-mode-setup doesn't setup
indentation when treesit-simple-indent-rules is *not* set, even when
treesit-indent-function has been set.

> I’ve now fixed that.

Indeed, thank you.

> (Though I didn’t expect someone to use this variable.)

Why not? It is documented and the natural alternative when
treesit-simple-indent-rules is not fit for the job, isn't it?

> And to Óscar, I’m curious and interested to know what indentation
> function you came up with; if it’s useful maybe we can add it to
> Emacs?

I started with treesit-simple-indent-rules until I stumbled on something
that it is not easily workable with the existing functions. My language
has a Lisp syntax, and the problem was with indenting comments (and with
how the presence of comments affected the indentation of other things).

For instance, the typical indentation of

(if p
    foo
  bar)

That's simple enough to achieve with treesit-simple-indent-rules, but if
we insert a comment:

(if p
    ;; blah
    (foo)
  (bar))

things suddenly turn hairy.

Likewise, starting with this:

(progn (foo)
       (bar))

... after inserting a comment, this is the desired indentation:

(progn ;; blah
  (foo)
  (bar))

... not this:

(progn ;; blah
       (foo)
       (bar))

Maybe a variant of `match' and `nth-sibling' that take an argument of
node types to skip would be enough to cover this case? That was
basically what I did.

Anyways, going the treesit-indent-function route was easy enough, and it
is more powerful, flexible and future-proof. I don't think that
treesit-simple-indent-rules would yield a significantly shorter and
mantainable implementation.

> I also updated the manual.

Thanks!

BTW, the treesit-based implementation of the major mode for this simple
language is a tiny fraction of its traditional counterpart. And
providing a great basis for adding more features :-)

bug#71086: 30.0.50; Setting treesit-indent-function is not effective is treesit-simple-indent-rules is nil

[Óscar Fuentes]

As the issues reported here are solved, I'm closing the bug.