bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]


Hi Eli and Yuan!

I've added a function to treesit.el, for use in
treesit-major-mode-setup.  Because the treesit-defun-type-regexp gives
us "defuns" for free, we can set fill-paragraph-function to this new
function, thus enabling quick formatting, and some sensible default for
fill-paragraph.  It aims to mirror c-indent-defun.

I think this is a nice way to get this functionality for free, but I'm
not 100% whether this is considered ok or not.

An alternative could be to add a 'treesit-mode-map' where we can
auto-enable such constructs.


Feel free to apply if it looks ok.

Theo



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-treesit-indent-defun.patch --]
[-- Type: text/x-diff, Size: 2035 bytes --]

From aed3f136d530c0eaa83c385cac7388c853b4f99a Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Mon, 28 Nov 2022 20:19:20 +0100
Subject: [PATCH] Add treesit--indent-defun

We can piggyback on the already existing 'treesit-defun-type-regexp'
to decide a scope to reindent/reformat.  By setting
fill-paragraph-function major modes that implement
'treesit-defun-type-regexp' will get this functionality for free, by
default bound to M-q.

* lisp/treesit.el (treesit--indent-defun): New function.
(treesit-major-mode-setup): Enable this functionality when prereqs are
fulfilled.
---
 lisp/treesit.el | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 8f092f475d..685c59aedb 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1451,6 +1451,21 @@ treesit-simple-indent
                         (message "No matched rule"))
                       (cons nil nil))))))
 
+(defun treesit--indent-defun (&optional _arg)
+  "Indent the current defun.
+
+Uses `treesit-defun-type-regexp' to determine which nodes to
+reindent."
+  (interactive)
+  (when-let ((defun-node
+              (save-excursion
+                (treesit-search-forward-goto
+                 (treesit-node-at (point))
+                 treesit-defun-type-regexp t t))))
+    (treesit-indent-region
+     (treesit-node-start defun-node)
+     (treesit-node-end defun-node))))
+
 (defun treesit-check-indent (mode)
   "Check current buffer's indentation against a major mode MODE.
 
@@ -1698,7 +1713,10 @@ treesit-major-mode-setup
   ;; Navigation.
   (when treesit-defun-type-regexp
     (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
-    (setq-local end-of-defun-function #'treesit-end-of-defun)))
+    (setq-local end-of-defun-function #'treesit-end-of-defun))
+  ;; Filling
+  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
+    (setq-local fill-paragraph-function #'treesit--indent-defun)))
 
 ;;; Debugging
 
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 67 bytes --]


Rebased on newest emacs-29 and added bug number in commit.

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-treesit-indent-defun.patch --]
[-- Type: text/x-diff, Size: 2049 bytes --]

From 0a39367e16849ca10775ffbe08c2929f95df1470 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Mon, 28 Nov 2022 20:19:20 +0100
Subject: [PATCH] Add treesit--indent-defun

We can piggyback on the already existing 'treesit-defun-type-regexp'
to decide a scope to reindent/reformat.  By setting
fill-paragraph-function major modes that implement
'treesit-defun-type-regexp' will get this functionality for free, by
default bound to M-q.

* lisp/treesit.el (treesit--indent-defun): New function.
(treesit-major-mode-setup): Enable this functionality when prereqs are
fulfilled.

 (bug#59662)
---
 lisp/treesit.el | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 8f092f475d..685c59aedb 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1451,6 +1451,21 @@ treesit-simple-indent
                         (message "No matched rule"))
                       (cons nil nil))))))
 
+(defun treesit--indent-defun (&optional _arg)
+  "Indent the current defun.
+
+Uses `treesit-defun-type-regexp' to determine which nodes to
+reindent."
+  (interactive)
+  (when-let ((defun-node
+              (save-excursion
+                (treesit-search-forward-goto
+                 (treesit-node-at (point))
+                 treesit-defun-type-regexp t t))))
+    (treesit-indent-region
+     (treesit-node-start defun-node)
+     (treesit-node-end defun-node))))
+
 (defun treesit-check-indent (mode)
   "Check current buffer's indentation against a major mode MODE.
 
@@ -1698,7 +1713,10 @@ treesit-major-mode-setup
   ;; Navigation.
   (when treesit-defun-type-regexp
     (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
-    (setq-local end-of-defun-function #'treesit-end-of-defun)))
+    (setq-local end-of-defun-function #'treesit-end-of-defun))
+  ;; Filling
+  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
+    (setq-local fill-paragraph-function #'treesit--indent-defun)))
 
 ;;; Debugging
 
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Cc: eliz@gnu.org, casouri@gmail.org
> From: Theodor Thornhill <theo@thornhill.no>
> Date: Mon, 28 Nov 2022 20:32:01 +0100
> 
> I've added a function to treesit.el, for use in
> treesit-major-mode-setup.  Because the treesit-defun-type-regexp gives
> us "defuns" for free, we can set fill-paragraph-function to this new
> function, thus enabling quick formatting, and some sensible default for
> fill-paragraph.  It aims to mirror c-indent-defun.
> 
> I think this is a nice way to get this functionality for free, but I'm
> not 100% whether this is considered ok or not.

Sounds good, but why did you think it wouldn't be OK?  Anything here that
doesn't meet the eye?

> An alternative could be to add a 'treesit-mode-map' where we can
> auto-enable such constructs.

I think this is less desirable.

Yuan, WDYT?

> @@ -1698,7 +1713,10 @@ treesit-major-mode-setup
>    ;; Navigation.
>    (when treesit-defun-type-regexp
>      (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
> -    (setq-local end-of-defun-function #'treesit-end-of-defun)))
> +    (setq-local end-of-defun-function #'treesit-end-of-defun))
> +  ;; Filling
> +  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
> +    (setq-local fill-paragraph-function #'treesit--indent-defun)))

I'm a bit confused: if the function's name is treesit--indent-defun, and it
uses treesit-indent-region to do its job, why do we assign it to
fill-paragraph-function, which is supposed to _fill_, not to _indent_?

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: eliz@gnu.org, casouri@gmail.org
>> From: Theodor Thornhill <theo@thornhill.no>
>> Date: Mon, 28 Nov 2022 20:32:01 +0100
>> 
>> I've added a function to treesit.el, for use in
>> treesit-major-mode-setup.  Because the treesit-defun-type-regexp gives
>> us "defuns" for free, we can set fill-paragraph-function to this new
>> function, thus enabling quick formatting, and some sensible default for
>> fill-paragraph.  It aims to mirror c-indent-defun.
>> 
>> I think this is a nice way to get this functionality for free, but I'm
>> not 100% whether this is considered ok or not.
>
> Sounds good, but why did you think it wouldn't be OK?  Anything here that
> doesn't meet the eye?
>

Not really, but see below answer.

>> An alternative could be to add a 'treesit-mode-map' where we can
>> auto-enable such constructs.
>
> I think this is less desirable.
>
> Yuan, WDYT?
>
>> @@ -1698,7 +1713,10 @@ treesit-major-mode-setup
>>    ;; Navigation.
>>    (when treesit-defun-type-regexp
>>      (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
>> -    (setq-local end-of-defun-function #'treesit-end-of-defun)))
>> +    (setq-local end-of-defun-function #'treesit-end-of-defun))
>> +  ;; Filling
>> +  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
>> +    (setq-local fill-paragraph-function #'treesit--indent-defun)))
>
> I'm a bit confused: if the function's name is treesit--indent-defun, and it
> uses treesit-indent-region to do its job, why do we assign it to
> fill-paragraph-function, which is supposed to _fill_, not to _indent_?

This is why I was thinking it would maybe be better to put it into a
treesit-mode-map that major-modes can inherit from, thus binding it to
things such as C-c C-q.  The reason I put it in filling was because that
is a common key to press in everything _but_ prog-modes.  And prog-modes
seem to mostly just turn it off if not inside of comments etc.  This
would behave just like that, except we would reformat/reindent/refill
code.

In a way filling _is_ formatting/reindenting, at least that's how I look
at it.

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

Resending with correct address of Yuan.

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: 59662@debbugs.gnu.org, casouri@gmail.org
> Date: Tue, 29 Nov 2022 13:14:21 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Cc: eliz@gnu.org, casouri@gmail.org
> >> From: Theodor Thornhill <theo@thornhill.no>
> >> Date: Mon, 28 Nov 2022 20:32:01 +0100
> >> 
> >> I've added a function to treesit.el, for use in
> >> treesit-major-mode-setup.  Because the treesit-defun-type-regexp gives
> >> us "defuns" for free, we can set fill-paragraph-function to this new
> >> function, thus enabling quick formatting, and some sensible default for
> >> fill-paragraph.  It aims to mirror c-indent-defun.
> >> 
> >> I think this is a nice way to get this functionality for free, but I'm
> >> not 100% whether this is considered ok or not.
> >
> > Sounds good, but why did you think it wouldn't be OK?  Anything here that
> > doesn't meet the eye?
> >
> 
> Not really, but see below answer.
> 
> >> An alternative could be to add a 'treesit-mode-map' where we can
> >> auto-enable such constructs.
> >
> > I think this is less desirable.
> >
> > Yuan, WDYT?
> >
> >> @@ -1698,7 +1713,10 @@ treesit-major-mode-setup
> >>    ;; Navigation.
> >>    (when treesit-defun-type-regexp
> >>      (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
> >> -    (setq-local end-of-defun-function #'treesit-end-of-defun)))
> >> +    (setq-local end-of-defun-function #'treesit-end-of-defun))
> >> +  ;; Filling
> >> +  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
> >> +    (setq-local fill-paragraph-function #'treesit--indent-defun)))
> >
> > I'm a bit confused: if the function's name is treesit--indent-defun, and it
> > uses treesit-indent-region to do its job, why do we assign it to
> > fill-paragraph-function, which is supposed to _fill_, not to _indent_?
> 
> This is why I was thinking it would maybe be better to put it into a
> treesit-mode-map that major-modes can inherit from, thus binding it to
> things such as C-c C-q.  The reason I put it in filling was because that
> is a common key to press in everything _but_ prog-modes.  And prog-modes
> seem to mostly just turn it off if not inside of comments etc.  This
> would behave just like that, except we would reformat/reindent/refill
> code.
> 
> In a way filling _is_ formatting/reindenting, at least that's how I look
> at it.
> 
> Theo
> 

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Yuan Fu]

Eli Zaretskii <eliz@gnu.org> writes:

> Resending with correct address of Yuan.
>
>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: 59662@debbugs.gnu.org, casouri@gmail.org
>> Date: Tue, 29 Nov 2022 13:14:21 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Cc: eliz@gnu.org, casouri@gmail.org
>> >> From: Theodor Thornhill <theo@thornhill.no>
>> >> Date: Mon, 28 Nov 2022 20:32:01 +0100
>> >> 
>> >> I've added a function to treesit.el, for use in
>> >> treesit-major-mode-setup.  Because the treesit-defun-type-regexp gives
>> >> us "defuns" for free, we can set fill-paragraph-function to this new
>> >> function, thus enabling quick formatting, and some sensible default for
>> >> fill-paragraph.  It aims to mirror c-indent-defun.
>> >> 
>> >> I think this is a nice way to get this functionality for free, but I'm
>> >> not 100% whether this is considered ok or not.
>> >
>> > Sounds good, but why did you think it wouldn't be OK?  Anything here that
>> > doesn't meet the eye?
>> >
>> 
>> Not really, but see below answer.
>> 
>> >> An alternative could be to add a 'treesit-mode-map' where we can
>> >> auto-enable such constructs.
>> >
>> > I think this is less desirable.
>> >
>> > Yuan, WDYT?
>> >
>> >> @@ -1698,7 +1713,10 @@ treesit-major-mode-setup
>> >>    ;; Navigation.
>> >>    (when treesit-defun-type-regexp
>> >>      (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
>> >> -    (setq-local end-of-defun-function #'treesit-end-of-defun)))
>> >> +    (setq-local end-of-defun-function #'treesit-end-of-defun))
>> >> +  ;; Filling
>> >> +  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
>> >> +    (setq-local fill-paragraph-function #'treesit--indent-defun)))
>> >
>> > I'm a bit confused: if the function's name is treesit--indent-defun, and it
>> > uses treesit-indent-region to do its job, why do we assign it to
>> > fill-paragraph-function, which is supposed to _fill_, not to _indent_?
>> 
>> This is why I was thinking it would maybe be better to put it into a
>> treesit-mode-map that major-modes can inherit from, thus binding it to
>> things such as C-c C-q.  The reason I put it in filling was because that
>> is a common key to press in everything _but_ prog-modes.  And prog-modes
>> seem to mostly just turn it off if not inside of comments etc.  This
>> would behave just like that, except we would reformat/reindent/refill
>> code.
>> 
>> In a way filling _is_ formatting/reindenting, at least that's how I look
>> at it.
>> 
>> Theo
>> 

I was caught up in some stuff the past two days, sorry about that :-)
I’ll have a look tomorrow!

Yuan

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Yuan Fu]

Theodor Thornhill <theo@thornhill.no> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> Cc: eliz@gnu.org, casouri@gmail.org
>>> From: Theodor Thornhill <theo@thornhill.no>
>>> Date: Mon, 28 Nov 2022 20:32:01 +0100
>>> 
>>> I've added a function to treesit.el, for use in
>>> treesit-major-mode-setup.  Because the treesit-defun-type-regexp gives
>>> us "defuns" for free, we can set fill-paragraph-function to this new
>>> function, thus enabling quick formatting, and some sensible default for
>>> fill-paragraph.  It aims to mirror c-indent-defun.
>>> 
>>> I think this is a nice way to get this functionality for free, but I'm
>>> not 100% whether this is considered ok or not.
>>
>> Sounds good, but why did you think it wouldn't be OK?  Anything here that
>> doesn't meet the eye?
>>
>
> Not really, but see below answer.
>
>>> An alternative could be to add a 'treesit-mode-map' where we can
>>> auto-enable such constructs.
>>
>> I think this is less desirable.
>>
>> Yuan, WDYT?
>>
>>> @@ -1698,7 +1713,10 @@ treesit-major-mode-setup
>>>    ;; Navigation.
>>>    (when treesit-defun-type-regexp
>>>      (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
>>> -    (setq-local end-of-defun-function #'treesit-end-of-defun)))
>>> +    (setq-local end-of-defun-function #'treesit-end-of-defun))
>>> +  ;; Filling
>>> +  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
>>> +    (setq-local fill-paragraph-function #'treesit--indent-defun)))
>>
>> I'm a bit confused: if the function's name is treesit--indent-defun, and it
>> uses treesit-indent-region to do its job, why do we assign it to
>> fill-paragraph-function, which is supposed to _fill_, not to _indent_?
>
> This is why I was thinking it would maybe be better to put it into a
> treesit-mode-map that major-modes can inherit from, thus binding it to
> things such as C-c C-q.  The reason I put it in filling was because that
> is a common key to press in everything _but_ prog-modes.  And prog-modes
> seem to mostly just turn it off if not inside of comments etc.  This
> would behave just like that, except we would reformat/reindent/refill
> code.
>
> In a way filling _is_ formatting/reindenting, at least that's how I look
> at it.

I see, so you want to implement C-c C-q in c-mode. But why don’t we
make treesit--indent-defun a command and bind it in C-c C-q in major modes? 
Filling is not indent IMO. If you fill a list in elisp code it wraps long lines 
instead of indent (IIRC).

Also, this is really not tree-sitter specific, it doesn’t require any
tree-sitter feature to accomplish: indent-defun only needs mark-defun
and indent-region, both are supported by practically any major mode.

Normally this kind of thing goes into lisp.el, alongside commands like
fill-paragraph, indent-region, beginning-of-defun, etc, and claim a
global keybinding. But maybe we only want it to live under C-c prefix,
in that case I guess we can bind it in prog-mode-map, under C-c C-q?

I’m thinking something like

(defun prog-mode-indent-defun ()
  (interactive)
  (mark-defun)
  (indent-region (region-beginning) (region-end)))

(defvar prog-mode-map
  (...
   (define-key map (kbd "C-c C-q") #'prog-mode-indent-defun)))

Yuan

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Yuan Fu]

> On Nov 30, 2022, at 3:23 PM, Yuan Fu <casouri@gmail.com> wrote:
> 
> 
> Theodor Thornhill <theo@thornhill.no> writes:
> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>>>> Cc: eliz@gnu.org, casouri@gmail.org
>>>> From: Theodor Thornhill <theo@thornhill.no>
>>>> Date: Mon, 28 Nov 2022 20:32:01 +0100
>>>> 
>>>> I've added a function to treesit.el, for use in
>>>> treesit-major-mode-setup.  Because the treesit-defun-type-regexp gives
>>>> us "defuns" for free, we can set fill-paragraph-function to this new
>>>> function, thus enabling quick formatting, and some sensible default for
>>>> fill-paragraph.  It aims to mirror c-indent-defun.
>>>> 
>>>> I think this is a nice way to get this functionality for free, but I'm
>>>> not 100% whether this is considered ok or not.
>>> 
>>> Sounds good, but why did you think it wouldn't be OK?  Anything here that
>>> doesn't meet the eye?
>>> 
>> 
>> Not really, but see below answer.
>> 
>>>> An alternative could be to add a 'treesit-mode-map' where we can
>>>> auto-enable such constructs.
>>> 
>>> I think this is less desirable.
>>> 
>>> Yuan, WDYT?
>>> 
>>>> @@ -1698,7 +1713,10 @@ treesit-major-mode-setup
>>>>   ;; Navigation.
>>>>   (when treesit-defun-type-regexp
>>>>     (setq-local beginning-of-defun-function #'treesit-beginning-of-defun)
>>>> -    (setq-local end-of-defun-function #'treesit-end-of-defun)))
>>>> +    (setq-local end-of-defun-function #'treesit-end-of-defun))
>>>> +  ;; Filling
>>>> +  (when (and treesit-defun-type-regexp treesit-simple-indent-rules)
>>>> +    (setq-local fill-paragraph-function #'treesit--indent-defun)))
>>> 
>>> I'm a bit confused: if the function's name is treesit--indent-defun, and it
>>> uses treesit-indent-region to do its job, why do we assign it to
>>> fill-paragraph-function, which is supposed to _fill_, not to _indent_?
>> 
>> This is why I was thinking it would maybe be better to put it into a
>> treesit-mode-map that major-modes can inherit from, thus binding it to
>> things such as C-c C-q.  The reason I put it in filling was because that
>> is a common key to press in everything _but_ prog-modes.  And prog-modes
>> seem to mostly just turn it off if not inside of comments etc.  This
>> would behave just like that, except we would reformat/reindent/refill
>> code.
>> 
>> In a way filling _is_ formatting/reindenting, at least that's how I look
>> at it.
> 
> I see, so you want to implement C-c C-q in c-mode. But why don’t we
> make treesit--indent-defun a command and bind it in C-c C-q in major modes? 
> Filling is not indent IMO. If you fill a list in elisp code it wraps long lines 
> instead of indent (IIRC).
> 
> Also, this is really not tree-sitter specific, it doesn’t require any
> tree-sitter feature to accomplish: indent-defun only needs mark-defun
> and indent-region, both are supported by practically any major mode.
> 
> Normally this kind of thing goes into lisp.el, alongside commands like
> fill-paragraph, indent-region, beginning-of-defun, etc, and claim a
> global keybinding. But maybe we only want it to live under C-c prefix,
> in that case I guess we can bind it in prog-mode-map, under C-c C-q?
> 
> I’m thinking something like
> 
> (defun prog-mode-indent-defun ()
>  (interactive)
>  (mark-defun)
>  (indent-region (region-beginning) (region-end)))
> 
> (defvar prog-mode-map
>  (...
>   (define-key map (kbd "C-c C-q") #'prog-mode-indent-defun)))

Tho I’d prefer we bind it in major mode maps first. Adding anything to prog-mode-map should go through emacs-devel (and I don’t know if we want to add any bindings to prog-mode-map), of course.

Yuan

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Yuan Fu <casouri@gmail.com> writes:

> Theodor Thornhill <theo@thornhill.no> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> Cc: eliz@gnu.org, casouri@gmail.org
>>>> From: Theodor Thornhill <theo@thornhill.no>
>>>> Date: Mon, 28 Nov 2022 20:32:01 +0100

[...]

>>
>>>> An alternative could be to add a 'treesit-mode-map' where we can
>>>> auto-enable such constructs.
>>>
>>> I think this is less desirable.
>>>
>>> Yuan, WDYT?
>>>
> I see, so you want to implement C-c C-q in c-mode. But why don’t we
> make treesit--indent-defun a command and bind it in C-c C-q in major modes? 
> Filling is not indent IMO. If you fill a list in elisp code it wraps long lines 
> instead of indent (IIRC).
>
> Also, this is really not tree-sitter specific, it doesn’t require any
> tree-sitter feature to accomplish: indent-defun only needs mark-defun
> and indent-region, both are supported by practically any major mode.
>
> Normally this kind of thing goes into lisp.el, alongside commands like
> fill-paragraph, indent-region, beginning-of-defun, etc, and claim a
> global keybinding. But maybe we only want it to live under C-c prefix,
> in that case I guess we can bind it in prog-mode-map, under C-c C-q?
>

Yeah, that's smart - thanks for the tip!  Perhaps adding it to prog-mode
is a bit premature, but maybe Eli has an opinion here?

My guess is that a change in lisp.el is a little too big for the release
branch?  So we could let this live in treesit.el if it's too big a
change?

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 30 Nov 2022 15:23:17 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  59662@debbugs.gnu.org,
>  casouri@gmail.org
> 
> Also, this is really not tree-sitter specific, it doesn’t require any
> tree-sitter feature to accomplish: indent-defun only needs mark-defun
> and indent-region, both are supported by practically any major mode.
> 
> Normally this kind of thing goes into lisp.el, alongside commands like
> fill-paragraph, indent-region, beginning-of-defun, etc, and claim a
> global keybinding. But maybe we only want it to live under C-c prefix,
> in that case I guess we can bind it in prog-mode-map, under C-c C-q?
> 
> I’m thinking something like
> 
> (defun prog-mode-indent-defun ()
>   (interactive)
>   (mark-defun)
>   (indent-region (region-beginning) (region-end)))
> 
> (defvar prog-mode-map
>   (...
>    (define-key map (kbd "C-c C-q") #'prog-mode-indent-defun)))

SGTM, but I'd begin by doing this only in c-ts-mode.el (and maybe also in
csharp-ts-mode).  After all, until now this functionality and the "C-c C-q"
binding was only in CC Mode.  (My original question about this was due to
lack of enthusiasm to add the "C Indent" node of the Emacs manual text that
explains this feature is only available in CC Mode, not in the tree-sitter
based C/C++ modes.)

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 30 Nov 2022 17:07:48 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  59662@debbugs.gnu.org,
>  casouri@gmail.org
> 
> > I’m thinking something like
> > 
> > (defun prog-mode-indent-defun ()
> >  (interactive)
> >  (mark-defun)
> >  (indent-region (region-beginning) (region-end)))
> > 
> > (defvar prog-mode-map
> >  (...
> >   (define-key map (kbd "C-c C-q") #'prog-mode-indent-defun)))
> 
> Tho I’d prefer we bind it in major mode maps first. Adding anything to prog-mode-map should go through emacs-devel (and I don’t know if we want to add any bindings to prog-mode-map), of course.

I agree and think we should first do this only in C and related modes.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 1 December 2022 08:13:22 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 30 Nov 2022 15:23:17 -0800
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>>  59662@debbugs.gnu.org,
>>  casouri@gmail.org
>> 
>> Also, this is really not tree-sitter specific, it doesn’t require any
>> tree-sitter feature to accomplish: indent-defun only needs mark-defun
>> and indent-region, both are supported by practically any major mode.
>> 
>> Normally this kind of thing goes into lisp.el, alongside commands like
>> fill-paragraph, indent-region, beginning-of-defun, etc, and claim a
>> global keybinding. But maybe we only want it to live under C-c prefix,
>> in that case I guess we can bind it in prog-mode-map, under C-c C-q?
>> 
>> I’m thinking something like
>> 
>> (defun prog-mode-indent-defun ()
>>   (interactive)
>>   (mark-defun)
>>   (indent-region (region-beginning) (region-end)))
>> 
>> (defvar prog-mode-map
>>   (...
>>    (define-key map (kbd "C-c C-q") #'prog-mode-indent-defun)))
>
>SGTM, but I'd begin by doing this only in c-ts-mode.el (and maybe also in
>csharp-ts-mode).  After all, until now this functionality and the "C-c C-q"
>binding was only in CC Mode.  (My original question about this was due to
>lack of enthusiasm to add the "C Indent" node of the Emacs manual text that
>explains this feature is only available in CC Mode, not in the tree-sitter
>based C/C++ modes.)

So you'd like the change in lisp.el, but bound only in the modes you mention, or have something similar to the original patch in treesit, but still bound only to the same modes?

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Thu, 01 Dec 2022 08:22:45 +0100
> From: Theodor Thornhill <theo@thornhill.no>
> CC: 59662@debbugs.gnu.org, casouri@gmail.org
> 
> >SGTM, but I'd begin by doing this only in c-ts-mode.el (and maybe also in
> >csharp-ts-mode).  After all, until now this functionality and the "C-c C-q"
> >binding was only in CC Mode.  (My original question about this was due to
> >lack of enthusiasm to add the "C Indent" node of the Emacs manual text that
> >explains this feature is only available in CC Mode, not in the tree-sitter
> >based C/C++ modes.)
> 
> So you'd like the change in lisp.el, but bound only in the modes you mention, or have something similar to the original patch in treesit, but still bound only to the same modes?

Why in lisp.el? we are talking about a feature of C-like languages.  The
current binding of "C-c C-q" is in cc-cmds.el, so what I had in mind is to
have its counterpart in c-ts-mode.el.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 01 Dec 2022 08:22:45 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: 59662@debbugs.gnu.org, casouri@gmail.org
>> 
>> >SGTM, but I'd begin by doing this only in c-ts-mode.el (and maybe also in
>> >csharp-ts-mode).  After all, until now this functionality and the "C-c C-q"
>> >binding was only in CC Mode.  (My original question about this was due to
>> >lack of enthusiasm to add the "C Indent" node of the Emacs manual text that
>> >explains this feature is only available in CC Mode, not in the tree-sitter
>> >based C/C++ modes.)
>> 
>> So you'd like the change in lisp.el, but bound only in the modes you
>> mention, or have something similar to the original patch in treesit,
>> but still bound only to the same modes?
>
> Why in lisp.el? we are talking about a feature of C-like languages.  The
> current binding of "C-c C-q" is in cc-cmds.el, so what I had in mind is to
> have its counterpart in c-ts-mode.el.

Only because Yuan mentioned it.  I will make that change.

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1005 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 01 Dec 2022 08:22:45 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: 59662@debbugs.gnu.org, casouri@gmail.org
>> 
>> >SGTM, but I'd begin by doing this only in c-ts-mode.el (and maybe also in
>> >csharp-ts-mode).  After all, until now this functionality and the "C-c C-q"
>> >binding was only in CC Mode.  (My original question about this was due to
>> >lack of enthusiasm to add the "C Indent" node of the Emacs manual text that
>> >explains this feature is only available in CC Mode, not in the tree-sitter
>> >based C/C++ modes.)
>> 
>> So you'd like the change in lisp.el, but bound only in the modes you mention, or have something similar to the original patch in treesit, but still bound only to the same modes?
>
> Why in lisp.el? we are talking about a feature of C-like languages.  The
> current binding of "C-c C-q" is in cc-cmds.el, so what I had in mind is to
> have its counterpart in c-ts-mode.el.

Something like this?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-c-ts-mode-indent-defun.patch --]
[-- Type: text/x-diff, Size: 1456 bytes --]

From 26a8780950017a911bac7290366da05e0e35f13f Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 2 Dec 2022 16:05:35 +0100
Subject: [PATCH] Add c-ts-mode--indent-defun

Add in this function to mimic 'c-indent-defun'

* lisp/progmodes/c-ts-mode.el (c-ts-mode--indent-defun): New function.
(c-ts-mode-map): New mode map that uses said function.
---
 lisp/progmodes/c-ts-mode.el | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index fcabb5beac..3171600901 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -518,9 +518,25 @@ c-ts-mode--end-of-defun
       (if (looking-at "\\s<\\|\n")
 	  (forward-line 1)))))
 
+(defun c-ts-mode--indent-defun ()
+  "Indent the current top-level declaration syntactically.
+
+`treesit-defun-type-regexp' defines what constructs to indent."
+  (interactive)
+  (save-excursion
+    (mark-defun)
+    (indent-region (region-beginning) (region-end))))
+
+(defvar-keymap c-ts-mode-map
+  :doc "Keymap for the C language with tree-sitter"
+  :parent prog-mode-map
+  "C-c C-q" #'c-ts-mode--indent-defun)
+
 ;;;###autoload
 (define-derived-mode c-ts-base-mode prog-mode "C"
-  "Major mode for editing C, powered by tree-sitter."
+  "Major mode for editing C, powered by tree-sitter.
+
+\\{c-ts-mode-map}"
   :syntax-table c-ts-mode--syntax-table
 
   ;; Navigation.
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: casouri@gmail.com, 59662@debbugs.gnu.org
> Date: Fri, 02 Dec 2022 16:06:46 +0100
> 
> > Why in lisp.el? we are talking about a feature of C-like languages.  The
> > current binding of "C-c C-q" is in cc-cmds.el, so what I had in mind is to
> > have its counterpart in c-ts-mode.el.
> 
> Something like this?

Yes, thanks.

But a command should not have "--" in its name, since every command is by
definition a public function, not an internal one.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: casouri@gmail.com, 59662@debbugs.gnu.org
>> Date: Fri, 02 Dec 2022 16:06:46 +0100
>> 
>> > Why in lisp.el? we are talking about a feature of C-like languages.  The
>> > current binding of "C-c C-q" is in cc-cmds.el, so what I had in mind is to
>> > have its counterpart in c-ts-mode.el.
>> 
>> Something like this?
>
> Yes, thanks.
>
> But a command should not have "--" in its name, since every command is by
> definition a public function, not an internal one.

That makes sense.  I was in doubt about that, but decided to go the
(wrong) direction.  Updated in latest patch.

However - mark-defun signals an error on some constructs because of
number-or-marker error:

Debugger entered--Lisp error: (wrong-type-argument integer-or-marker-p nil)
  treesit-end-of-defun()
  c-ts-mode--end-of-defun()
  end-of-defun()
  mark-defun(1 nil)
  mark-defun(1 23209)
  funcall-interactively(mark-defun 1 23209)
  call-interactively(mark-defun record nil)
  command-execute(mark-defun record)
  execute-extended-command(nil "mark-defun" nil)
  funcall-interactively(execute-extended-command nil "mark-defun" nil)
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

I'll file a separate bug report for this one, if that's ok?  Yuan, are
you ok with this patch?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-c-ts-mode-indent-defun.patch --]
[-- Type: text/x-diff, Size: 1452 bytes --]

From b7c82cf11404a68338aa495cbd4424f45e1f8405 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 2 Dec 2022 16:05:35 +0100
Subject: [PATCH] Add c-ts-mode-indent-defun

Add in this function to mimic 'c-indent-defun'

* lisp/progmodes/c-ts-mode.el (c-ts-mode-indent-defun): New function.
(c-ts-mode-map): New mode map that uses said function.
---
 lisp/progmodes/c-ts-mode.el | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index fcabb5beac..36d67c5701 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -518,9 +518,25 @@ c-ts-mode--end-of-defun
       (if (looking-at "\\s<\\|\n")
 	  (forward-line 1)))))
 
+(defun c-ts-mode-indent-defun ()
+  "Indent the current top-level declaration syntactically.
+
+`treesit-defun-type-regexp' defines what constructs to indent."
+  (interactive)
+  (save-excursion
+    (mark-defun)
+    (indent-region (region-beginning) (region-end))))
+
+(defvar-keymap c-ts-mode-map
+  :doc "Keymap for the C language with tree-sitter"
+  :parent prog-mode-map
+  "C-c C-q" #'c-ts-mode-indent-defun)
+
 ;;;###autoload
 (define-derived-mode c-ts-base-mode prog-mode "C"
-  "Major mode for editing C, powered by tree-sitter."
+  "Major mode for editing C, powered by tree-sitter.
+
+\\{c-ts-mode-map}"
   :syntax-table c-ts-mode--syntax-table
 
   ;; Navigation.
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: casouri@gmail.com, 59662@debbugs.gnu.org
> Date: Fri, 02 Dec 2022 19:32:33 +0100
> 
> However - mark-defun signals an error on some constructs because of
> number-or-marker error:
> 
> Debugger entered--Lisp error: (wrong-type-argument integer-or-marker-p nil)
>   treesit-end-of-defun()
>   c-ts-mode--end-of-defun()
>   end-of-defun()
>   mark-defun(1 nil)
>   mark-defun(1 23209)
>   funcall-interactively(mark-defun 1 23209)
>   call-interactively(mark-defun record nil)
>   command-execute(mark-defun record)
>   execute-extended-command(nil "mark-defun" nil)
>   funcall-interactively(execute-extended-command nil "mark-defun" nil)
>   call-interactively(execute-extended-command nil nil)
>   command-execute(execute-extended-command)
> 
> I'll file a separate bug report for this one, if that's ok?

Yes, please.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Daniel Martín via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Theodor Thornhill via "Bug reports for GNU Emacs, the Swiss army knife
of text editors" <bug-gnu-emacs@gnu.org> writes:

>>
>> Why in lisp.el? we are talking about a feature of C-like languages.  The
>> current binding of "C-c C-q" is in cc-cmds.el, so what I had in mind is to
>> have its counterpart in c-ts-mode.el.
>
> Something like this?

Thanks, some comments below:

>
> From 26a8780950017a911bac7290366da05e0e35f13f Mon Sep 17 00:00:00 2001
> From: Theodor Thornhill <theo@thornhill.no>
> Date: Fri, 2 Dec 2022 16:05:35 +0100
> Subject: [PATCH] Add c-ts-mode--indent-defun
>
> Add in this function to mimic 'c-indent-defun'
>
> * lisp/progmodes/c-ts-mode.el (c-ts-mode--indent-defun): New function.
> (c-ts-mode-map): New mode map that uses said function.
> ---
>  lisp/progmodes/c-ts-mode.el | 18 +++++++++++++++++-
>  1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
> index fcabb5beac..3171600901 100644
> --- a/lisp/progmodes/c-ts-mode.el
> +++ b/lisp/progmodes/c-ts-mode.el
> @@ -518,9 +518,25 @@ c-ts-mode--end-of-defun
>        (if (looking-at "\\s<\\|\n")
>  	  (forward-line 1)))))
>  
> +(defun c-ts-mode--indent-defun ()

This should be public: c-ts-mode-indent-defun

> +  "Indent the current top-level declaration syntactically.
> +
> +`treesit-defun-type-regexp' defines what constructs to indent."
> +  (interactive)

(interactive "*") will handle read-only buffers a bit better.

> +  (save-excursion
> +    (mark-defun)
> +    (indent-region (region-beginning) (region-end))))

Nit: Once you call (mark-defun), you can call

(indent-region (point) (mark))

which should be more efficient than calling (region-beginning) or
(region-end).

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 01/12/2022 09:13, Eli Zaretskii wrote:
> After all, until now this functionality and the "C-c C-q"
> binding was only in CC Mode.

There is also the 'M-q' binding in the popular paredit-mode.

It runs paredit-reindent-defun.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 02/12/2022 21:53, Dmitry Gutov wrote:
> On 01/12/2022 09:13, Eli Zaretskii wrote:
>> After all, until now this functionality and the "C-c C-q"
>> binding was only in CC Mode.
> 
> There is also the 'M-q' binding in the popular paredit-mode.
> 
> It runs paredit-reindent-defun.

That function runs fill-paragraph when in string or comment, and 
beginning-of-defun + indent-sexp otherwise.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 2 December 2022 20:53:13 CET, Dmitry Gutov <dgutov@yandex.ru> wrote:
>On 01/12/2022 09:13, Eli Zaretskii wrote:
>> After all, until now this functionality and the "C-c C-q"
>> binding was only in CC Mode.
>
>There is also the 'M-q' binding in the popular paredit-mode.
>
>It runs paredit-reindent-defun.

Yeah. My first iteration was piggybacking on fill-paragraph-function to get that binding, but I believe that was undesirable.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2459 bytes --]

Daniel Martín <mardani29@yahoo.es> writes:

> Theodor Thornhill via "Bug reports for GNU Emacs, the Swiss army knife
> of text editors" <bug-gnu-emacs@gnu.org> writes:
>
>>>
>>> Why in lisp.el? we are talking about a feature of C-like languages.  The
>>> current binding of "C-c C-q" is in cc-cmds.el, so what I had in mind is to
>>> have its counterpart in c-ts-mode.el.
>>
>> Something like this?
>
> Thanks, some comments below:
>

Thanks for looking!

>>
>> From 26a8780950017a911bac7290366da05e0e35f13f Mon Sep 17 00:00:00 2001
>> From: Theodor Thornhill <theo@thornhill.no>
>> Date: Fri, 2 Dec 2022 16:05:35 +0100
>> Subject: [PATCH] Add c-ts-mode--indent-defun
>>
>> Add in this function to mimic 'c-indent-defun'
>>
>> * lisp/progmodes/c-ts-mode.el (c-ts-mode--indent-defun): New function.
>> (c-ts-mode-map): New mode map that uses said function.
>> ---
>>  lisp/progmodes/c-ts-mode.el | 18 +++++++++++++++++-
>>  1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
>> index fcabb5beac..3171600901 100644
>> --- a/lisp/progmodes/c-ts-mode.el
>> +++ b/lisp/progmodes/c-ts-mode.el
>> @@ -518,9 +518,25 @@ c-ts-mode--end-of-defun
>>        (if (looking-at "\\s<\\|\n")
>>  	  (forward-line 1)))))
>>  
>> +(defun c-ts-mode--indent-defun ()
>
> This should be public: c-ts-mode-indent-defun
>

Yeah.

>> +  "Indent the current top-level declaration syntactically.
>> +
>> +`treesit-defun-type-regexp' defines what constructs to indent."
>> +  (interactive)
>
> (interactive "*") will handle read-only buffers a bit better.
>

I was wondering about that, but I couldn't really find any situation
that gave any difference.  But I've added it now.

>> +  (save-excursion
>> +    (mark-defun)
>> +    (indent-region (region-beginning) (region-end))))
>
> Nit: Once you call (mark-defun), you can call
>
> (indent-region (point) (mark))
>
> which should be more efficient than calling (region-beginning) or
> (region-end).

Right - thanks.  However, this makes me wonder - should we really be
setting mark here? I see that c-indent-defun does not, and it feels
weird that indenting adds to the mark ring.  What do you think?  My
first patch used

(indent-region
  (treesit-node-start node)
  (treesit-node-end node))

Which behaves similarly to c-indent-defun.

See attached patch.

Thanks,

Theo



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-c-ts-mode-indent-defun.patch --]
[-- Type: text/x-diff, Size: 1439 bytes --]

From 9c22569f7a98c9fd671664e0834bd079e8f20bd3 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 2 Dec 2022 16:05:35 +0100
Subject: [PATCH] Add c-ts-mode-indent-defun

Add in this function to mimic 'c-indent-defun'

* lisp/progmodes/c-ts-mode.el (c-ts-mode-indent-defun): New function.
(c-ts-mode-map): New mode map that uses said function.
---
 lisp/progmodes/c-ts-mode.el | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index fcabb5beac..677ff07a2d 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -518,9 +518,25 @@ c-ts-mode--end-of-defun
       (if (looking-at "\\s<\\|\n")
 	  (forward-line 1)))))
 
+(defun c-ts-mode-indent-defun ()
+  "Indent the current top-level declaration syntactically.
+
+`treesit-defun-type-regexp' defines what constructs to indent."
+  (interactive "*")
+  (save-excursion
+    (mark-defun)
+    (indent-region (point) (mark))))
+
+(defvar-keymap c-ts-mode-map
+  :doc "Keymap for the C language with tree-sitter"
+  :parent prog-mode-map
+  "C-c C-q" #'c-ts-mode-indent-defun)
+
 ;;;###autoload
 (define-derived-mode c-ts-base-mode prog-mode "C"
-  "Major mode for editing C, powered by tree-sitter."
+  "Major mode for editing C, powered by tree-sitter.
+
+\\{c-ts-mode-map}"
   :syntax-table c-ts-mode--syntax-table
 
   ;; Navigation.
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Daniel Martín via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Theodor Thornhill <theo@thornhill.no> writes:

> Right - thanks.  However, this makes me wonder - should we really be
> setting mark here? I see that c-indent-defun does not, and it feels
> weird that indenting adds to the mark ring.  What do you think?  My
> first patch used
>
> (indent-region
>   (treesit-node-start node)
>   (treesit-node-end node))
>
> Which behaves similarly to c-indent-defun.
>
> See attached patch.

I don't have a strong opinion, but if we can indent without setting the
mark, I think it'd be a cleaner command.  Specially if c-indent-defun
doesn't set the mark either.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>Theodor Thornhill <theo@thornhill.no> writes:
>
>> Right - thanks.  However, this makes me wonder - should we really be
>> setting mark here? I see that c-indent-defun does not, and it feels
>> weird that indenting adds to the mark ring.  What do you think?  My
>> first patch used
>>
>> (indent-region
>>   (treesit-node-start node)
>>   (treesit-node-end node))
>>
>> Which behaves similarly to c-indent-defun.
>>
>> See attached patch.
>
>I don't have a strong opinion, but if we can indent without setting the
>mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>doesn't set the mark either.

Yeah, we don't need to set the mark. Thanks for the feedback :)
Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 02/12/2022 22:07, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> 
> On 2 December 2022 20:53:13 CET, Dmitry Gutov<dgutov@yandex.ru>  wrote:
>> On 01/12/2022 09:13, Eli Zaretskii wrote:
>>> After all, until now this functionality and the "C-c C-q"
>>> binding was only in CC Mode.
>> There is also the 'M-q' binding in the popular paredit-mode.
>>
>> It runs paredit-reindent-defun.
> Yeah. My first iteration was piggybacking on fill-paragraph-function to get that binding, but I believe that was undesirable.

IME it's quite handy to have in modes where there is separation between 
(rigid) code and strings/comments.

Not sure if that's all of prog-mode descendants, or if there will be 
exceptions.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Sat, 3 Dec 2022 00:27:51 +0200
> Cc: 59662@debbugs.gnu.org, casouri@gmail.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 02/12/2022 22:07, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
> > 
> > On 2 December 2022 20:53:13 CET, Dmitry Gutov<dgutov@yandex.ru>  wrote:
> >> On 01/12/2022 09:13, Eli Zaretskii wrote:
> >>> After all, until now this functionality and the "C-c C-q"
> >>> binding was only in CC Mode.
> >> There is also the 'M-q' binding in the popular paredit-mode.
> >>
> >> It runs paredit-reindent-defun.
> > Yeah. My first iteration was piggybacking on fill-paragraph-function to get that binding, but I believe that was undesirable.
> 
> IME it's quite handy to have in modes where there is separation between 
> (rigid) code and strings/comments.
> 
> Not sure if that's all of prog-mode descendants, or if there will be 
> exceptions.

I'm okay with making "C-c C-q" work in more modes than just C-related ones,
but please do that on master, not on the release branch.

As for M-q, it should be bound to a fill-SOMETHING command, not to a
indent-SOMETHING command.  So from my POV paredit-mode should be fixed to
use some other sequence for paredit-reindent-defun, perhaps even "C-c C-q",
but that is IMO an unrelated issue.

And if you wanted to suggest that an indent-defun command in c-ts-mode be
bound to M-q, then I'm very much against that: M-q already has a useful
command bound to it in CC Mode, and should do something similar in
c-ts-mode; I believe we are discussing that in bug#59763.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 03/12/2022 08:47, Eli Zaretskii wrote:
>> Date: Sat, 3 Dec 2022 00:27:51 +0200
>> Cc: 59662@debbugs.gnu.org, casouri@gmail.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> On 02/12/2022 22:07, Theodor Thornhill via Bug reports for GNU Emacs,
>> the Swiss army knife of text editors wrote:
>>>
>>> On 2 December 2022 20:53:13 CET, Dmitry Gutov<dgutov@yandex.ru>  wrote:
>>>> On 01/12/2022 09:13, Eli Zaretskii wrote:
>>>>> After all, until now this functionality and the "C-c C-q"
>>>>> binding was only in CC Mode.
>>>> There is also the 'M-q' binding in the popular paredit-mode.
>>>>
>>>> It runs paredit-reindent-defun.
>>> Yeah. My first iteration was piggybacking on fill-paragraph-function to get that binding, but I believe that was undesirable.
>>
>> IME it's quite handy to have in modes where there is separation between
>> (rigid) code and strings/comments.
>>
>> Not sure if that's all of prog-mode descendants, or if there will be
>> exceptions.
> 
> I'm okay with making "C-c C-q" work in more modes than just C-related ones,
> but please do that on master, not on the release branch.
> 
> As for M-q, it should be bound to a fill-SOMETHING command, not to a
> indent-SOMETHING command.  So from my POV paredit-mode should be fixed to
> use some other sequence for paredit-reindent-defun, perhaps even "C-c C-q",
> but that is IMO an unrelated issue.
> 
> And if you wanted to suggest that an indent-defun command in c-ts-mode be
> bound to M-q, then I'm very much against that: M-q already has a useful
> command bound to it in CC Mode, and should do something similar in
> c-ts-mode; I believe we are discussing that in bug#59763.

Both c-fill-paragraph and the proposed ts-c-fill-paragraph perform 
fill-region when inside a string or comment. Otherwise, do nothing.

paredit-reindent-defun, like I described, does all that too. But when 
outside strings and comments, it reindents the current defun.

AFAICT, these behaviors are quite compatible. But paredit's binding 
seems to be useful in more contexts.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Sat, 3 Dec 2022 17:59:04 +0200
> Cc: theo@thornhill.no, casouri@gmail.com, 59662@debbugs.gnu.org,
>  casouri@gmail.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > As for M-q, it should be bound to a fill-SOMETHING command, not to a
> > indent-SOMETHING command.  So from my POV paredit-mode should be fixed to
> > use some other sequence for paredit-reindent-defun, perhaps even "C-c C-q",
> > but that is IMO an unrelated issue.
> > 
> > And if you wanted to suggest that an indent-defun command in c-ts-mode be
> > bound to M-q, then I'm very much against that: M-q already has a useful
> > command bound to it in CC Mode, and should do something similar in
> > c-ts-mode; I believe we are discussing that in bug#59763.
> 
> Both c-fill-paragraph and the proposed ts-c-fill-paragraph perform 
> fill-region when inside a string or comment. Otherwise, do nothing.

Yes.

> paredit-reindent-defun, like I described, does all that too. But when 
> outside strings and comments, it reindents the current defun.

So maybe paredit-reindent-defun doesn't need any fixes, but using it as a
role model for the C-like languages is not TRT, since in C these are two
separate commands with two different key bindings.

> AFAICT, these behaviors are quite compatible. But paredit's binding 
> seems to be useful in more contexts.

I'm fine with trying something like that on master, to see whether users
like this conflation, but not on the release branch.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Theodor Thornhill <theo@thornhill.no> writes:

> On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>>Theodor Thornhill <theo@thornhill.no> writes:
>>
>>> Right - thanks.  However, this makes me wonder - should we really be
>>> setting mark here? I see that c-indent-defun does not, and it feels
>>> weird that indenting adds to the mark ring.  What do you think?  My
>>> first patch used
>>>
>>> (indent-region
>>>   (treesit-node-start node)
>>>   (treesit-node-end node))
>>>
>>> Which behaves similarly to c-indent-defun.
>>>
>>> See attached patch.
>>
>>I don't have a strong opinion, but if we can indent without setting the
>>mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>>doesn't set the mark either.
>
> Yeah, we don't need to set the mark. Thanks for the feedback :)
> Theo

Eli, do you have any opinion on this matter? Should we implement this
function without using marks?  That will make the function a smidge
bigger, but would not mess up the mark-ring.  I think that's desirable,
at least.

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: "Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife
>  of text editors" <bug-gnu-emacs@gnu.org>, Eli Zaretskii <eliz@gnu.org>,
>  casouri@gmail.com, 59662@debbugs.gnu.org
> Date: Sun, 04 Dec 2022 13:33:35 +0100
> 
> Theodor Thornhill <theo@thornhill.no> writes:
> 
> > On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
> >>Theodor Thornhill <theo@thornhill.no> writes:
> >>
> >>> Right - thanks.  However, this makes me wonder - should we really be
> >>> setting mark here? I see that c-indent-defun does not, and it feels
> >>> weird that indenting adds to the mark ring.  What do you think?  My
> >>> first patch used
> >>>
> >>> (indent-region
> >>>   (treesit-node-start node)
> >>>   (treesit-node-end node))
> >>>
> >>> Which behaves similarly to c-indent-defun.
> >>>
> >>> See attached patch.
> >>
> >>I don't have a strong opinion, but if we can indent without setting the
> >>mark, I think it'd be a cleaner command.  Specially if c-indent-defun
> >>doesn't set the mark either.
> >
> > Yeah, we don't need to set the mark. Thanks for the feedback :)
> > Theo
> 
> Eli, do you have any opinion on this matter? Should we implement this
> function without using marks?  That will make the function a smidge
> bigger, but would not mess up the mark-ring.  I think that's desirable,
> at least.

IMO, it is better not to set the mark, indeed.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1748 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: "Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife
>>  of text editors" <bug-gnu-emacs@gnu.org>, Eli Zaretskii <eliz@gnu.org>,
>>  casouri@gmail.com, 59662@debbugs.gnu.org
>> Date: Sun, 04 Dec 2022 13:33:35 +0100
>> 
>> Theodor Thornhill <theo@thornhill.no> writes:
>> 
>> > On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>> >>Theodor Thornhill <theo@thornhill.no> writes:
>> >>
>> >>> Right - thanks.  However, this makes me wonder - should we really be
>> >>> setting mark here? I see that c-indent-defun does not, and it feels
>> >>> weird that indenting adds to the mark ring.  What do you think?  My
>> >>> first patch used
>> >>>
>> >>> (indent-region
>> >>>   (treesit-node-start node)
>> >>>   (treesit-node-end node))
>> >>>
>> >>> Which behaves similarly to c-indent-defun.
>> >>>
>> >>> See attached patch.
>> >>
>> >>I don't have a strong opinion, but if we can indent without setting the
>> >>mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>> >>doesn't set the mark either.
>> >
>> > Yeah, we don't need to set the mark. Thanks for the feedback :)
>> > Theo
>> 
>> Eli, do you have any opinion on this matter? Should we implement this
>> function without using marks?  That will make the function a smidge
>> bigger, but would not mess up the mark-ring.  I think that's desirable,
>> at least.
>
> IMO, it is better not to set the mark, indeed.

Agreed, see attached patch.  What do you think, Yuan and Eli?

Should we expose the internal helper I used, or maybe this function
could just as well live in treesit.c?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-c-ts-mode-indent-defun.patch --]
[-- Type: text/x-diff, Size: 1737 bytes --]

From 7638a3b0c4c15800f70d0d7e1f444d7512e8994f Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 2 Dec 2022 16:05:35 +0100
Subject: [PATCH] Add c-ts-mode-indent-defun

Add in this function to mimic 'c-indent-defun'

* lisp/progmodes/c-ts-mode.el (c-ts-mode-indent-defun): New function.
(c-ts-mode-map): New mode map that uses said function.

(bug#59662)
---
 lisp/progmodes/c-ts-mode.el | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 0c66b4959e..998e988901 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -517,9 +517,30 @@ c-ts-mode--end-of-defun
       (if (looking-at "\\s<\\|\n")
 	  (forward-line 1)))))
 
+(defun c-ts-mode-indent-defun ()
+  "Indent the current top-level declaration syntactically.
+
+`treesit-defun-type-regexp' defines what constructs to indent."
+  (interactive "*")
+  (when-let ((node (treesit-search-forward
+                    (treesit-node-at (point))
+                    treesit-defun-type-regexp t t)))
+    ;; Use internal function to make sure we get the top-level node.
+    (setq node (treesit--defun-maybe-top-level node))
+    (indent-region
+     (treesit-node-start node)
+     (treesit-node-end node))))
+
+(defvar-keymap c-ts-mode-map
+  :doc "Keymap for the C language with tree-sitter"
+  :parent prog-mode-map
+  "C-c C-q" #'c-ts-mode-indent-defun)
+
 ;;;###autoload
 (define-derived-mode c-ts-base-mode prog-mode "C"
-  "Major mode for editing C, powered by tree-sitter."
+  "Major mode for editing C, powered by tree-sitter.
+
+\\{c-ts-mode-map}"
   :syntax-table c-ts-mode--syntax-table
 
   ;; Navigation.
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: mardani29@yahoo.es, bug-gnu-emacs@gnu.org, casouri@gmail.com,
>  59662@debbugs.gnu.org
> Date: Wed, 07 Dec 2022 10:31:13 +0100
> 
> > IMO, it is better not to set the mark, indeed.
> 
> Agreed, see attached patch.  What do you think, Yuan and Eli?
> 
> Should we expose the internal helper I used, or maybe this function
> could just as well live in treesit.c?

If it is expected to be useful outside treesit.el, I think it should
become public.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Yuan Fu]

Theodor Thornhill <theo@thornhill.no> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Theodor Thornhill <theo@thornhill.no>
>>> Cc: "Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife
>>>  of text editors" <bug-gnu-emacs@gnu.org>, Eli Zaretskii <eliz@gnu.org>,
>>>  casouri@gmail.com, 59662@debbugs.gnu.org
>>> Date: Sun, 04 Dec 2022 13:33:35 +0100
>>> 
>>> Theodor Thornhill <theo@thornhill.no> writes:
>>> 
>>> > On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>>> >>Theodor Thornhill <theo@thornhill.no> writes:
>>> >>
>>> >>> Right - thanks.  However, this makes me wonder - should we really be
>>> >>> setting mark here? I see that c-indent-defun does not, and it feels
>>> >>> weird that indenting adds to the mark ring.  What do you think?  My
>>> >>> first patch used
>>> >>>
>>> >>> (indent-region
>>> >>>   (treesit-node-start node)
>>> >>>   (treesit-node-end node))
>>> >>>
>>> >>> Which behaves similarly to c-indent-defun.
>>> >>>
>>> >>> See attached patch.
>>> >>
>>> >>I don't have a strong opinion, but if we can indent without setting the
>>> >>mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>>> >>doesn't set the mark either.
>>> >
>>> > Yeah, we don't need to set the mark. Thanks for the feedback :)
>>> > Theo
>>> 
>>> Eli, do you have any opinion on this matter? Should we implement this
>>> function without using marks?  That will make the function a smidge
>>> bigger, but would not mess up the mark-ring.  I think that's desirable,
>>> at least.
>>
>> IMO, it is better not to set the mark, indeed.
>
> Agreed, see attached patch.  What do you think, Yuan and Eli?
>
> Should we expose the internal helper I used, or maybe this function
> could just as well live in treesit.c?
>
> Theo
>

Very nice, I applied this patch with some changes to the function. I
think using treesit-beginning/end-of-defun is more robust.

Yuan

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Yuan Fu <casouri@gmail.com> writes:

> Theodor Thornhill <theo@thornhill.no> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Theodor Thornhill <theo@thornhill.no>
>>>> Cc: "Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife
>>>>  of text editors" <bug-gnu-emacs@gnu.org>, Eli Zaretskii <eliz@gnu.org>,
>>>>  casouri@gmail.com, 59662@debbugs.gnu.org
>>>> Date: Sun, 04 Dec 2022 13:33:35 +0100
>>>> 
>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>> 
>>>> > On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>>>> >>Theodor Thornhill <theo@thornhill.no> writes:
>>>> >>
>>>> >>> Right - thanks.  However, this makes me wonder - should we really be
>>>> >>> setting mark here? I see that c-indent-defun does not, and it feels
>>>> >>> weird that indenting adds to the mark ring.  What do you think?  My
>>>> >>> first patch used
>>>> >>>
>>>> >>> (indent-region
>>>> >>>   (treesit-node-start node)
>>>> >>>   (treesit-node-end node))
>>>> >>>
>>>> >>> Which behaves similarly to c-indent-defun.
>>>> >>>
>>>> >>> See attached patch.
>>>> >>
>>>> >>I don't have a strong opinion, but if we can indent without setting the
>>>> >>mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>>>> >>doesn't set the mark either.
>>>> >
>>>> > Yeah, we don't need to set the mark. Thanks for the feedback :)
>>>> > Theo
>>>> 
>>>> Eli, do you have any opinion on this matter? Should we implement this
>>>> function without using marks?  That will make the function a smidge
>>>> bigger, but would not mess up the mark-ring.  I think that's desirable,
>>>> at least.
>>>
>>> IMO, it is better not to set the mark, indeed.
>>
>> Agreed, see attached patch.  What do you think, Yuan and Eli?
>>
>> Should we expose the internal helper I used, or maybe this function
>> could just as well live in treesit.c?
>>
>> Theo
>>
>
> Very nice, I applied this patch with some changes to the function. I
> think using treesit-beginning/end-of-defun is more robust.
>
> Yuan

Thanks!  Would you mind describing why?

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Yuan Fu]

> On Dec 7, 2022, at 12:06 PM, Theodor Thornhill <theo@thornhill.no> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>> Theodor Thornhill <theo@thornhill.no> writes:
>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>>>> From: Theodor Thornhill <theo@thornhill.no>
>>>>> Cc: "Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife
>>>>> of text editors" <bug-gnu-emacs@gnu.org>, Eli Zaretskii <eliz@gnu.org>,
>>>>> casouri@gmail.com, 59662@debbugs.gnu.org
>>>>> Date: Sun, 04 Dec 2022 13:33:35 +0100
>>>>> 
>>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>>> 
>>>>>> On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>>>>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>>>>> 
>>>>>>>> Right - thanks.  However, this makes me wonder - should we really be
>>>>>>>> setting mark here? I see that c-indent-defun does not, and it feels
>>>>>>>> weird that indenting adds to the mark ring.  What do you think?  My
>>>>>>>> first patch used
>>>>>>>> 
>>>>>>>> (indent-region
>>>>>>>>  (treesit-node-start node)
>>>>>>>>  (treesit-node-end node))
>>>>>>>> 
>>>>>>>> Which behaves similarly to c-indent-defun.
>>>>>>>> 
>>>>>>>> See attached patch.
>>>>>>> 
>>>>>>> I don't have a strong opinion, but if we can indent without setting the
>>>>>>> mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>>>>>>> doesn't set the mark either.
>>>>>> 
>>>>>> Yeah, we don't need to set the mark. Thanks for the feedback :)
>>>>>> Theo
>>>>> 
>>>>> Eli, do you have any opinion on this matter? Should we implement this
>>>>> function without using marks?  That will make the function a smidge
>>>>> bigger, but would not mess up the mark-ring.  I think that's desirable,
>>>>> at least.
>>>> 
>>>> IMO, it is better not to set the mark, indeed.
>>> 
>>> Agreed, see attached patch.  What do you think, Yuan and Eli?
>>> 
>>> Should we expose the internal helper I used, or maybe this function
>>> could just as well live in treesit.c?
>>> 
>>> Theo
>>> 
>> 
>> Very nice, I applied this patch with some changes to the function. I
>> think using treesit-beginning/end-of-defun is more robust.
>> 
>> Yuan
> 
> Thanks!  Would you mind describing why?

Yes, sorry. I’m planning to change treesit-beginning-of-defun and treesit-end-of-defun so they can behave differently according to user configuration, basically they can decide whether to skip nested defuns or not. If we use these functions rather than re-implementing their logic (to some extent) in c-ts-mode-indent-defun, any improvement to defun navigation would automatically benefit c-ts-mode-indent-defun.

Yuan

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 7 December 2022 22:40:31 CET, Yuan Fu <casouri@gmail.com> wrote:
>
>
>> On Dec 7, 2022, at 12:06 PM, Theodor Thornhill <theo@thornhill.no> wrote:
>> 
>> Yuan Fu <casouri@gmail.com> writes:
>> 
>>> Theodor Thornhill <theo@thornhill.no> writes:
>>> 
>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>> 
>>>>>> From: Theodor Thornhill <theo@thornhill.no>
>>>>>> Cc: "Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife
>>>>>> of text editors" <bug-gnu-emacs@gnu.org>, Eli Zaretskii <eliz@gnu.org>,
>>>>>> casouri@gmail.com, 59662@debbugs.gnu.org
>>>>>> Date: Sun, 04 Dec 2022 13:33:35 +0100
>>>>>> 
>>>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>>>> 
>>>>>>> On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>>>>>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>>>>>> 
>>>>>>>>> Right - thanks.  However, this makes me wonder - should we really be
>>>>>>>>> setting mark here? I see that c-indent-defun does not, and it feels
>>>>>>>>> weird that indenting adds to the mark ring.  What do you think?  My
>>>>>>>>> first patch used
>>>>>>>>> 
>>>>>>>>> (indent-region
>>>>>>>>>  (treesit-node-start node)
>>>>>>>>>  (treesit-node-end node))
>>>>>>>>> 
>>>>>>>>> Which behaves similarly to c-indent-defun.
>>>>>>>>> 
>>>>>>>>> See attached patch.
>>>>>>>> 
>>>>>>>> I don't have a strong opinion, but if we can indent without setting the
>>>>>>>> mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>>>>>>>> doesn't set the mark either.
>>>>>>> 
>>>>>>> Yeah, we don't need to set the mark. Thanks for the feedback :)
>>>>>>> Theo
>>>>>> 
>>>>>> Eli, do you have any opinion on this matter? Should we implement this
>>>>>> function without using marks?  That will make the function a smidge
>>>>>> bigger, but would not mess up the mark-ring.  I think that's desirable,
>>>>>> at least.
>>>>> 
>>>>> IMO, it is better not to set the mark, indeed.
>>>> 
>>>> Agreed, see attached patch.  What do you think, Yuan and Eli?
>>>> 
>>>> Should we expose the internal helper I used, or maybe this function
>>>> could just as well live in treesit.c?
>>>> 
>>>> Theo
>>>> 
>>> 
>>> Very nice, I applied this patch with some changes to the function. I
>>> think using treesit-beginning/end-of-defun is more robust.
>>> 
>>> Yuan
>> 
>> Thanks!  Would you mind describing why?
>
>Yes, sorry. I’m planning to change treesit-beginning-of-defun and treesit-end-of-defun so they can behave differently according to user configuration, basically they can decide whether to skip nested defuns or not. If we use these functions rather than re-implementing their logic (to some extent) in c-ts-mode-indent-defun, any improvement to defun navigation would automatically benefit c-ts-mode-indent-defun.
>
>Yuan

Thanks for explaining your rationale. 

By the way, there's an issue with the patch you applied:

void
foo()
{
  ...
}

If point is at the v in void, the current implementation will not reinvent. I didn't get time to debug yet, though. Possibly that will be fixed after your rework as well.

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Yuan Fu]

> On Dec 7, 2022, at 1:53 PM, Theodor Thornhill <theo@thornhill.no> wrote:
> 
> 
> 
> On 7 December 2022 22:40:31 CET, Yuan Fu <casouri@gmail.com> wrote:
>> 
>> 
>>> On Dec 7, 2022, at 12:06 PM, Theodor Thornhill <theo@thornhill.no> wrote:
>>> 
>>> Yuan Fu <casouri@gmail.com> writes:
>>> 
>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>> 
>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> 
>>>>>>> From: Theodor Thornhill <theo@thornhill.no>
>>>>>>> Cc: "Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife
>>>>>>> of text editors" <bug-gnu-emacs@gnu.org>, Eli Zaretskii <eliz@gnu.org>,
>>>>>>> casouri@gmail.com, 59662@debbugs.gnu.org
>>>>>>> Date: Sun, 04 Dec 2022 13:33:35 +0100
>>>>>>> 
>>>>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>>>>> 
>>>>>>>> On 2 December 2022 22:09:55 CET, "Daniel Martín" <mardani29@yahoo.es> wrote:
>>>>>>>>> Theodor Thornhill <theo@thornhill.no> writes:
>>>>>>>>> 
>>>>>>>>>> Right - thanks.  However, this makes me wonder - should we really be
>>>>>>>>>> setting mark here? I see that c-indent-defun does not, and it feels
>>>>>>>>>> weird that indenting adds to the mark ring.  What do you think?  My
>>>>>>>>>> first patch used
>>>>>>>>>> 
>>>>>>>>>> (indent-region
>>>>>>>>>> (treesit-node-start node)
>>>>>>>>>> (treesit-node-end node))
>>>>>>>>>> 
>>>>>>>>>> Which behaves similarly to c-indent-defun.
>>>>>>>>>> 
>>>>>>>>>> See attached patch.
>>>>>>>>> 
>>>>>>>>> I don't have a strong opinion, but if we can indent without setting the
>>>>>>>>> mark, I think it'd be a cleaner command.  Specially if c-indent-defun
>>>>>>>>> doesn't set the mark either.
>>>>>>>> 
>>>>>>>> Yeah, we don't need to set the mark. Thanks for the feedback :)
>>>>>>>> Theo
>>>>>>> 
>>>>>>> Eli, do you have any opinion on this matter? Should we implement this
>>>>>>> function without using marks?  That will make the function a smidge
>>>>>>> bigger, but would not mess up the mark-ring.  I think that's desirable,
>>>>>>> at least.
>>>>>> 
>>>>>> IMO, it is better not to set the mark, indeed.
>>>>> 
>>>>> Agreed, see attached patch.  What do you think, Yuan and Eli?
>>>>> 
>>>>> Should we expose the internal helper I used, or maybe this function
>>>>> could just as well live in treesit.c?
>>>>> 
>>>>> Theo
>>>>> 
>>>> 
>>>> Very nice, I applied this patch with some changes to the function. I
>>>> think using treesit-beginning/end-of-defun is more robust.
>>>> 
>>>> Yuan
>>> 
>>> Thanks!  Would you mind describing why?
>> 
>> Yes, sorry. I’m planning to change treesit-beginning-of-defun and treesit-end-of-defun so they can behave differently according to user configuration, basically they can decide whether to skip nested defuns or not. If we use these functions rather than re-implementing their logic (to some extent) in c-ts-mode-indent-defun, any improvement to defun navigation would automatically benefit c-ts-mode-indent-defun.
>> 
>> Yuan
> 
> Thanks for explaining your rationale. 
> 
> By the way, there's an issue with the patch you applied:
> 
> void
> foo()
> {
>  ...
> }
> 
> If point is at the v in void, the current implementation will not reinvent. I didn't get time to debug yet, though. Possibly that will be fixed after your rework as well.

Ah, thanks, I’ll fix that. And I’ll ask for your opinion before making changes next time, however innocent I thought the change is :-)

Yuan

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]

On 03/12/2022 19:54, Eli Zaretskii wrote:
>> paredit-reindent-defun, like I described, does all that too. But when
>> outside strings and comments, it reindents the current defun.
> So maybe paredit-reindent-defun doesn't need any fixes, but using it as a
> role model for the C-like languages is not TRT, since in C these are two
> separate commands with two different key bindings.

It's not a given that the current state of things must automatically be 
optimal. But of course I can't vouch whether this behavior is optimal 
for every language.

And major modes will continue to be able to rebind these key sequences.

>> AFAICT, these behaviors are quite compatible. But paredit's binding
>> seems to be useful in more contexts.
> I'm fine with trying something like that on master, to see whether users
> like this conflation, but not on the release branch.

Here's a patch that people can try. From my brief testing, seems useful 
enough. I suppose some might dislike it, though, in some codebases where 
the methods/functions are consistently longer than a screenful. I do 
like this binding in Emacs Lisp too, though.

[-- Attachment #2: prog-reindent-defun.diff --]
[-- Type: text/x-patch, Size: 1559 bytes --]

diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..46bacea737 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -102,7 +102,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -144,6 +145,26 @@ prog-first-column
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
 
+(defun prog-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (if (or (nth 8 (syntax-ppss))
+            (re-search-forward comment-start-skip (line-end-position) t))
+        (if (memq fill-paragraph-function '(t nil))
+            (lisp-fill-paragraph argument)
+          (funcall fill-paragraph-function argument))
+      (end-of-defun)
+      (let ((end (point)))
+        (beginning-of-defun)
+        (indent-region (point) end)))))
+
 (defvar-local prettify-symbols-alist nil
   "Alist of symbol prettifications.
 Each element looks like (SYMBOL . CHARACTER), where the symbol

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

  Would you mind describing why?
>>> 
>>> Yes, sorry. I’m planning to change treesit-beginning-of-defun and treesit-end-of-defun so they can behave differently according to user configuration, basically they can decide whether to skip nested defuns or not. If we use these functions rather than re-implementing their logic (to some extent) in c-ts-mode-indent-defun, any improvement to defun navigation would automatically benefit c-ts-mode-indent-defun.
>>> 
>>> Yuan
>> 
>> Thanks for explaining your rationale. 
>> 
>> By the way, there's an issue with the patch you applied:
>> 
>> void
>> foo()
>> {
>>  ...
>> }
>> 
>> If point is at the v in void, the current implementation will not reinvent. I didn't get time to debug yet, though. Possibly that will be fixed after your rework as well.
>
>Ah, thanks, I’ll fix that. And I’ll ask for your opinion before making changes next time, however innocent I thought the change is :-)
>
>Yuan

No stress, just wanted to point it out, but thanks!

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Thu, 8 Dec 2022 03:37:54 +0200
> Cc: theo@thornhill.no, casouri@gmail.com, 59662@debbugs.gnu.org,
>  casouri@gmail.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> >> AFAICT, these behaviors are quite compatible. But paredit's binding
> >> seems to be useful in more contexts.
> > I'm fine with trying something like that on master, to see whether users
> > like this conflation, but not on the release branch.
> 
> Here's a patch that people can try. From my brief testing, seems useful 
> enough. I suppose some might dislike it, though, in some codebases where 
> the methods/functions are consistently longer than a screenful. I do 
> like this binding in Emacs Lisp too, though.

Stefan, Lars: any comments?

I'm okay with installing this on master, but please rename the new
function to something like prog-fill-reindent; the "fill" part should
appear in the name, to explain why we have both prog-indent and this
function.

Also, a NEWS entry and some addition to the "Multi-line Indent" node
of the Emacs user manual would be nice.

Bonus points for adding something that could use treesit.el
capabilities, when those are available, instead of syntax-ppss, to
determine when we are in a comment or in a string.  Or maybe Theo or
Yuan could suggest such an addition.

Thanks.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 8 Dec 2022 03:37:54 +0200
>> Cc: theo@thornhill.no, casouri@gmail.com, 59662@debbugs.gnu.org,
>>  casouri@gmail.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> 
>> >> AFAICT, these behaviors are quite compatible. But paredit's binding
>> >> seems to be useful in more contexts.
>> > I'm fine with trying something like that on master, to see whether users
>> > like this conflation, but not on the release branch.
>> 
>> Here's a patch that people can try. From my brief testing, seems useful 
>> enough. I suppose some might dislike it, though, in some codebases where 
>> the methods/functions are consistently longer than a screenful. I do 
>> like this binding in Emacs Lisp too, though.
>
> Stefan, Lars: any comments?
>
> I'm okay with installing this on master, but please rename the new
> function to something like prog-fill-reindent; the "fill" part should
> appear in the name, to explain why we have both prog-indent and this
> function.
>
> Also, a NEWS entry and some addition to the "Multi-line Indent" node
> of the Emacs user manual would be nice.
>
> Bonus points for adding something that could use treesit.el
> capabilities, when those are available, instead of syntax-ppss, to
> determine when we are in a comment or in a string.  Or maybe Theo or
> Yuan could suggest such an addition.
>

This works perfectly as is in *-ts-modes.  At least for modes where
syntax-table is set correctly.  I don't think there's a reason to enable
treesit.el capabilities, unless there's a case where this doesn't work.

If this ends up in master, should I backport some similar feature for
the *ts-modes?

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: casouri@gmail.com, 59662@debbugs.gnu.org, casouri@gmail.org
> Date: Thu, 08 Dec 2022 09:11:28 +0100
> 
> > Bonus points for adding something that could use treesit.el
> > capabilities, when those are available, instead of syntax-ppss, to
> > determine when we are in a comment or in a string.  Or maybe Theo or
> > Yuan could suggest such an addition.
> 
> This works perfectly as is in *-ts-modes.  At least for modes where
> syntax-table is set correctly.  I don't think there's a reason to enable
> treesit.el capabilities, unless there's a case where this doesn't work.

My point is that tree-sitter in many cases parses the program better
than syntax-ppss.  So if we can use its information about comments and
strings, why not do that?  It's exactly the same logic that Dmitry
used:

> +    (if (or (nth 8 (syntax-ppss))
> +            (re-search-forward comment-start-skip (line-end-position) t))

which falls back on regexp search if syntax-ppss cannot help.  I'm
asking why not try tree-sitter before we fall back on syntax-ppss or
regexp search?

Does that make sense?  If not, why not?

> If this ends up in master, should I backport some similar feature for
> the *ts-modes?

Why do we need this on the release branch?  A general
indent-or-fill-comment feature for all the progmodes didn't exist
until now, which is why we intend to install this on master.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: casouri@gmail.com, 59662@debbugs.gnu.org, casouri@gmail.org
>> Date: Thu, 08 Dec 2022 09:11:28 +0100
>> 
>> > Bonus points for adding something that could use treesit.el
>> > capabilities, when those are available, instead of syntax-ppss, to
>> > determine when we are in a comment or in a string.  Or maybe Theo or
>> > Yuan could suggest such an addition.
>> 
>> This works perfectly as is in *-ts-modes.  At least for modes where
>> syntax-table is set correctly.  I don't think there's a reason to enable
>> treesit.el capabilities, unless there's a case where this doesn't work.
>
> My point is that tree-sitter in many cases parses the program better
> than syntax-ppss.  So if we can use its information about comments and
> strings, why not do that?  It's exactly the same logic that Dmitry
> used:
>

Something like this?  If we add a defvar such as the below, similar to
treesit-defun-type-regexp we can use that in the function below:

```
(defvar-local treesit-comment-type-regexp nil
  "A regexp that matches the node type of comment nodes.

For example, \"(line|block)_comment\". ")


(defun prog-reindent-defun (&optional argument)
  "Refill paragraph or reindent the definition that the point is on.

If the point is in a string, or in a comment, or there is a
comment on the current line, fill the paragraph that the point is
in or is on the same line.

Otherwise, reindent the definition around or below point."
  (interactive "P")
  (save-excursion
    (if (or (and (treesit-available-p)
                 (treesit-ready-p (treesit-language-at (point)))
                 (string-match-p
                  treesit-comment-type-regexp
                  (treesit-node-type (treesit-node-at (point)))))
            (nth 8 (syntax-ppss))
            (re-search-forward comment-start-skip (line-end-position) t))
        (if (memq fill-paragraph-function '(t nil))
            (lisp-fill-paragraph argument)
          (funcall fill-paragraph-function argument))
      (end-of-defun)
      (let ((end (point)))
        (beginning-of-defun)
        (indent-region (point) end)))))
```

The error handling here is naive, but is this something in the line of
what you're suggesting?  If so, I can tweak the error handling and
create a patch for it?  Unless you want to do that, Dmitry

>> +    (if (or (nth 8 (syntax-ppss))
>> +            (re-search-forward comment-start-skip (line-end-position) t))
>
> which falls back on regexp search if syntax-ppss cannot help.  I'm
> asking why not try tree-sitter before we fall back on syntax-ppss or
> regexp search?
>
> Does that make sense?  If not, why not?
>

It does.

>> If this ends up in master, should I backport some similar feature for
>> the *ts-modes?
>
> Why do we need this on the release branch?  A general
> indent-or-fill-comment feature for all the progmodes didn't exist
> until now, which is why we intend to install this on master.

Yeah, sure :-)

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Resending because I removed the wrong email to Yuan...

Theodor Thornhill <theo@thornhill.no> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Theodor Thornhill <theo@thornhill.no>
>>> Cc: casouri@gmail.com, 59662@debbugs.gnu.org, casouri@gmail.org
>>> Date: Thu, 08 Dec 2022 09:11:28 +0100
>>> 
>>> > Bonus points for adding something that could use treesit.el
>>> > capabilities, when those are available, instead of syntax-ppss, to
>>> > determine when we are in a comment or in a string.  Or maybe Theo or
>>> > Yuan could suggest such an addition.
>>> 
>>> This works perfectly as is in *-ts-modes.  At least for modes where
>>> syntax-table is set correctly.  I don't think there's a reason to enable
>>> treesit.el capabilities, unless there's a case where this doesn't work.
>>
>> My point is that tree-sitter in many cases parses the program better
>> than syntax-ppss.  So if we can use its information about comments and
>> strings, why not do that?  It's exactly the same logic that Dmitry
>> used:
>>
>
> Something like this?  If we add a defvar such as the below, similar to
> treesit-defun-type-regexp we can use that in the function below:
>
> ```
> (defvar-local treesit-comment-type-regexp nil
>   "A regexp that matches the node type of comment nodes.
>
> For example, \"(line|block)_comment\". ")
>
>
> (defun prog-reindent-defun (&optional argument)
>   "Refill paragraph or reindent the definition that the point is on.
>
> If the point is in a string, or in a comment, or there is a
> comment on the current line, fill the paragraph that the point is
> in or is on the same line.
>
> Otherwise, reindent the definition around or below point."
>   (interactive "P")
>   (save-excursion
>     (if (or (and (treesit-available-p)
>                  (treesit-ready-p (treesit-language-at (point)))
>                  (string-match-p
>                   treesit-comment-type-regexp
>                   (treesit-node-type (treesit-node-at (point)))))
>             (nth 8 (syntax-ppss))
>             (re-search-forward comment-start-skip (line-end-position) t))
>         (if (memq fill-paragraph-function '(t nil))
>             (lisp-fill-paragraph argument)
>           (funcall fill-paragraph-function argument))
>       (end-of-defun)
>       (let ((end (point)))
>         (beginning-of-defun)
>         (indent-region (point) end)))))
> ```
>
> The error handling here is naive, but is this something in the line of
> what you're suggesting?  If so, I can tweak the error handling and
> create a patch for it?  Unless you want to do that, Dmitry
>
>>> +    (if (or (nth 8 (syntax-ppss))
>>> +            (re-search-forward comment-start-skip (line-end-position) t))
>>
>> which falls back on regexp search if syntax-ppss cannot help.  I'm
>> asking why not try tree-sitter before we fall back on syntax-ppss or
>> regexp search?
>>
>> Does that make sense?  If not, why not?
>>
>
> It does.
>
>>> If this ends up in master, should I backport some similar feature for
>>> the *ts-modes?
>>
>> Why do we need this on the release branch?  A general
>> indent-or-fill-comment feature for all the progmodes didn't exist
>> until now, which is why we intend to install this on master.
>
> Yeah, sure :-)
>
> Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>  59662@debbugs.gnu.org, casouri@gmail.org
> Date: Thu, 08 Dec 2022 09:56:44 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > My point is that tree-sitter in many cases parses the program better
> > than syntax-ppss.  So if we can use its information about comments and
> > strings, why not do that?  It's exactly the same logic that Dmitry
> > used:
> >
> 
> Something like this?  If we add a defvar such as the below, similar to
> treesit-defun-type-regexp we can use that in the function below:
> 
> ```
> (defvar-local treesit-comment-type-regexp nil
>   "A regexp that matches the node type of comment nodes.
> 
> For example, \"(line|block)_comment\". ")

The intent is that major modes set this variable?  SGTM.

>     (if (or (and (treesit-available-p)
>                  (treesit-ready-p (treesit-language-at (point)))

I'd imagine that this kind of test should have an API, so the
treesit-ready-p call should not be made explicitly?  Yuan, WDYT?

Also, isn't it enough to check whether the buffer has a tree-sitter
parser or something?

> The error handling here is naive, but is this something in the line of
> what you're suggesting?

Yes, it is what I had in mind, but see above.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>>  59662@debbugs.gnu.org, casouri@gmail.org
>> Date: Thu, 08 Dec 2022 09:56:44 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > My point is that tree-sitter in many cases parses the program better
>> > than syntax-ppss.  So if we can use its information about comments and
>> > strings, why not do that?  It's exactly the same logic that Dmitry
>> > used:
>> >
>> 
>> Something like this?  If we add a defvar such as the below, similar to
>> treesit-defun-type-regexp we can use that in the function below:
>> 
>> ```
>> (defvar-local treesit-comment-type-regexp nil
>>   "A regexp that matches the node type of comment nodes.
>> 
>> For example, \"(line|block)_comment\". ")
>
> The intent is that major modes set this variable?  SGTM.

Yeah, that was the idea.

>
>>     (if (or (and (treesit-available-p)
>>                  (treesit-ready-p (treesit-language-at (point)))
>
> I'd imagine that this kind of test should have an API, so the
> treesit-ready-p call should not be made explicitly?  Yuan, WDYT?
>
> Also, isn't it enough to check whether the buffer has a tree-sitter
> parser or something?
>
>> The error handling here is naive, but is this something in the line of
>> what you're suggesting?
>
> Yes, it is what I had in mind, but see above.

What about this?  I changed it to make a point that we should fill if
inside of a string or comment, and indent if inside a code block.  We
one such variable for each type, or in both as suggested below.

```
(defvar-local treesit-text-type-regexp nil
  "A regexp that matches the node type of textual nodes.

For example, \"(line|block)_comment\" in the case of a comment,
or \"string_literal\" in the case of a string.")

(defun prog-reindent-defun (&optional argument)
  "Refill paragraph or reindent the definition that the point is on.

If the point is in a string, or in a comment, or there is a
comment on the current line, fill the paragraph that the point is
in or is on the same line.

Otherwise, reindent the definition around or below point."
  (interactive "P")
  (save-excursion
    (if (or (and (treesit-parser-list)
                 (string-match-p
                  treesit-text-type-regexp
                  (treesit-node-type (treesit-node-at (point)))))
            (nth 8 (syntax-ppss))
            (re-search-forward comment-start-skip (line-end-position) t))
        (if (memq fill-paragraph-function '(t nil))
            (lisp-fill-paragraph argument)
          (funcall fill-paragraph-function argument))
      (end-of-defun)
      (let ((end (point)))
        (beginning-of-defun)
        (indent-region (point) end)))))
```

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>  59662@debbugs.gnu.org, casouri@gmail.com
> Date: Thu, 08 Dec 2022 12:09:18 +0100
> 
> What about this?  I changed it to make a point that we should fill if
> inside of a string or comment, and indent if inside a code block.  We
> one such variable for each type, or in both as suggested below.

I'll defer to Dmitry on that.

Thanks.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 10:56, Theodor Thornhill wrote:
>      (if (or (and (treesit-available-p)
>                   (treesit-ready-p (treesit-language-at (point)))
>                   (string-match-p
>                    treesit-comment-type-regexp
>                    (treesit-node-type (treesit-node-at (point)))))

I was kind of hoping that sometime in the near future our tree-sitter 
code would somehow plug into syntax-ppss (maybe via advice or etc) to 
provide the same results faster.

Or that we'd add similar helpers which would dispatch to either 
syntax-ppss or tree-sitter, depending on whether the latter is enabled 
and working.

The first approach will be more compatible for third-party code to use 
right now; the second will probably be more powerful. We might even do both.

As a result, a lot of functions won't have to reimplement the quoted 
part. Inside, and (more importantly) outside Emacs core.

So my preference would be to check in the implementation as-is (if 
everybody agrees with the behavior), and then migrate it to new helpers 
(in case plugging into syntax-ppss won't be automatic).

But I don't really mind the code you posted, if somebody (e.g. Eli) is 
going to insist on it. It just doesn't seem critical from the 
performance POV, and the variable treesit-comment-type-regexp seems to 
be a little out of place. But then again, if the helpers will be put in 
prog-mode.el, the var will be there as well.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 13:09, Theodor Thornhill wrote:
> What about this?  I changed it to make a point that we should fill if
> inside of a string or comment, and indent if inside a code block.  We
> one such variable for each type, or in both as suggested below.

Could you rephrase maybe? I'm not seeing the difference.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 09:16, Eli Zaretskii wrote:
> I'm okay with installing this on master, but please rename the new
> function to something like prog-fill-reindent; the "fill" part should
> appear in the name, to explain why we have both prog-indent and this
> function.

Sure.

prog-indent-sexp is fairly different, though, so it wouldn't be out of 
the question to have both commands, even if prog-reindent-defun didn't 
refill.

> Also, a NEWS entry and some addition to the "Multi-line Indent" node
> of the Emacs user manual would be nice.

I suppose.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Thu, 8 Dec 2022 16:18:17 +0200
> Cc: monnier@iro.umontreal.ca, larsi@gnus.org, 59662@debbugs.gnu.org,
>  casouri@gmail.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 08/12/2022 10:56, Theodor Thornhill wrote:
> >      (if (or (and (treesit-available-p)
> >                   (treesit-ready-p (treesit-language-at (point)))
> >                   (string-match-p
> >                    treesit-comment-type-regexp
> >                    (treesit-node-type (treesit-node-at (point)))))
> 
> I was kind of hoping that sometime in the near future our tree-sitter 
> code would somehow plug into syntax-ppss (maybe via advice or etc) to 
> provide the same results faster.

It is too early to consider practical code along these lines, we don't
have enough experience with tree-sitter.  I'm not even sure it can be
easily factored into the same categories as syntax-ppss uses.

IMO, the way to move to a unified handling is to use tree-sitter
wherever we can, originally as special-cased code, and later refactor
that into generalized APIs, when we know more about how to define and
use them.

> So my preference would be to check in the implementation as-is (if 
> everybody agrees with the behavior), and then migrate it to new helpers 
> (in case plugging into syntax-ppss won't be automatic).
> 
> But I don't really mind the code you posted, if somebody (e.g. Eli) is 
> going to insist on it. It just doesn't seem critical from the 
> performance POV, and the variable treesit-comment-type-regexp seems to 
> be a little out of place. But then again, if the helpers will be put in 
> prog-mode.el, the var will be there as well.

I don't insist, but it sounds like the logical thing to do.  So unless
you feel strongly against that, I'd say let's add that part.  This is,
after all, an experiment, so let's experiment.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/12/2022 13:09, Theodor Thornhill wrote:
>> What about this?  I changed it to make a point that we should fill if
>> inside of a string or comment, and indent if inside a code block.  We
>> one such variable for each type, or in both as suggested below.
>
> Could you rephrase maybe? I'm not seeing the difference.

In the first example I made the variable specific to comments.  But
filling should work on everything that's considered text, IIUC.  So if
the variable refers to both the regex needs to accommodate strings and
comments.  That will in turn make it less useful if we want to find
comments specifically, or strings specifically. so we could have both
'treesit-comment-regexp' and 'treesit-string-regexp' and match on '(or
treesit-comment-regexp treesit-string-regexp)' in this particular case.

Not really sure if that made more sense.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 16:24, Dmitry Gutov wrote:
> On 08/12/2022 09:16, Eli Zaretskii wrote:
>> I'm okay with installing this on master, but please rename the new
>> function to something like prog-fill-reindent; the "fill" part should
>> appear in the name, to explain why we have both prog-indent and this
>> function.
> 
> Sure.

I think we'll need to cram the word "defun" in the name too, won't we?

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 16:53, Eli Zaretskii wrote:
>> Date: Thu, 8 Dec 2022 16:18:17 +0200
>> Cc: monnier@iro.umontreal.ca, larsi@gnus.org, 59662@debbugs.gnu.org,
>>   casouri@gmail.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> On 08/12/2022 10:56, Theodor Thornhill wrote:
>>>       (if (or (and (treesit-available-p)
>>>                    (treesit-ready-p (treesit-language-at (point)))
>>>                    (string-match-p
>>>                     treesit-comment-type-regexp
>>>                     (treesit-node-type (treesit-node-at (point)))))
>>
>> I was kind of hoping that sometime in the near future our tree-sitter
>> code would somehow plug into syntax-ppss (maybe via advice or etc) to
>> provide the same results faster.
> 
> It is too early to consider practical code along these lines, we don't
> have enough experience with tree-sitter.  I'm not even sure it can be
> easily factored into the same categories as syntax-ppss uses.

I'm fairly certain that tree-sitter contexts could be "dumbed down" into 
syntax-ppss categories.

But that's not the only way: another is to just use tree-sitter to find 
a "safe place" to call parse-partial-sexp from. That would give us the 
speed bump together with 100% backward compatibility.

> IMO, the way to move to a unified handling is to use tree-sitter
> wherever we can, originally as special-cased code, and later refactor
> that into generalized APIs, when we know more about how to define and
> use them.

syntax-ppss is used in a lot of places. A "free" improvement like the 
above wouldn't hurt.

>> So my preference would be to check in the implementation as-is (if
>> everybody agrees with the behavior), and then migrate it to new helpers
>> (in case plugging into syntax-ppss won't be automatic).
>>
>> But I don't really mind the code you posted, if somebody (e.g. Eli) is
>> going to insist on it. It just doesn't seem critical from the
>> performance POV, and the variable treesit-comment-type-regexp seems to
>> be a little out of place. But then again, if the helpers will be put in
>> prog-mode.el, the var will be there as well.
> 
> I don't insist, but it sounds like the logical thing to do.  So unless
> you feel strongly against that, I'd say let's add that part.  This is,
> after all, an experiment, so let's experiment.

I'm not going to object.

Doubly so if somebody else wants to finish up the patch and commit it 
with NEWS and documentation updates. ;-)

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Thu, 8 Dec 2022 17:37:55 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: casouri@gmail.com, 59662@debbugs.gnu.org, theo@thornhill.no,
>  casouri@gmail.org
> 
> On 08/12/2022 16:24, Dmitry Gutov wrote:
> > On 08/12/2022 09:16, Eli Zaretskii wrote:
> >> I'm okay with installing this on master, but please rename the new
> >> function to something like prog-fill-reindent; the "fill" part should
> >> appear in the name, to explain why we have both prog-indent and this
> >> function.
> > 
> > Sure.
> 
> I think we'll need to cram the word "defun" in the name too, won't we?

I won't object to that.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>
>syntax-ppss is used in a lot of places. A "free" improvement like the above wouldn't hurt.
>
>>> So my preference would be to check in the , the var will be there as well.
>> 
>> I don't insist, but it sounds like the logical thing to do.  So unless
>> you feel strongly against that, I'd say let's add that part.  This is,
>> after all, an experiment, so let's experiment.
>
>I'm not going to object.
>
>Doubly so if somebody else wants to finish up the patch and commit it with NEWS and documentation updates. ;-)

I can finish up the patch if you don't want to :)

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 16:58, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Dmitry Gutov<dgutov@yandex.ru>  writes:
> 
>> On 08/12/2022 13:09, Theodor Thornhill wrote:
>>> What about this?  I changed it to make a point that we should fill if
>>> inside of a string or comment, and indent if inside a code block.  We
>>> one such variable for each type, or in both as suggested below.
>> Could you rephrase maybe? I'm not seeing the difference.
> In the first example I made the variable specific to comments.  But
> filling should work on everything that's considered text, IIUC.  So if
> the variable refers to both the regex needs to accommodate strings and
> comments.  That will in turn make it less useful if we want to find
> comments specifically, or strings specifically. so we could have both
> 'treesit-comment-regexp' and 'treesit-string-regexp' and match on '(or
> treesit-comment-regexp treesit-string-regexp)' in this particular case.
> 
> Not really sure if that made more sense.

Ah. So that was about the name of the variable. Looks good.

Note that if the improvement which I described ever comes to life 
(hopefully), then the variable will probably need to be 
renamed/moved/multiplied again.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 18:04, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> 
> 
>> syntax-ppss is used in a lot of places. A "free" improvement like the above wouldn't hurt.
>>
>>>> So my preference would be to check in the , the var will be there as well.
>>> I don't insist, but it sounds like the logical thing to do.  So unless
>>> you feel strongly against that, I'd say let's add that part.  This is,
>>> after all, an experiment, so let's experiment.
>> I'm not going to object.
>>
>> Doubly so if somebody else wants to finish up the patch and commit it with NEWS and documentation updates. 😉
> I can finish up the patch if you don't want to 😄

If you don't mind. Thank you. :-)

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 8 December 2022 17:06:26 CET, Dmitry Gutov <dgutov@yandex.ru> wrote:
>On 08/12/2022 16:58, Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors wrote:
>> Dmitry Gutov<dgutov@yandex.ru>  writes:
>> 
>>> On 08/12/2022 13:09, Theodor Thornhill wrote:
>>>> What about this?  I changed it to make a point that we should fill if
>>>> inside of a string or comment, and indent if inside a code block.  We
>>>> one such variable for each type, or in both as suggested below.
>>> Could you rephrase maybe? I'm not seeing the difference.
>> In the first example I made the variable specific to comments.  But
>> filling should work on everything that's considered text, IIUC.  So if
>> the variable refers to both the regex needs to accommodate strings and
>> comments.  That will in turn make it less useful if we want to find
>> comments specifically, or strings specifically. so we could have both
>> 'treesit-comment-regexp' and 'treesit-string-regexp' and match on '(or
>> treesit-comment-regexp treesit-string-regexp)' in this particular case.
>> 
>> Not really sure if that made more sense.
>
>Ah. So that was about the name of the variable. Looks good.
>

Yeah, and thus what nodes it should capture


>Note that if the improvement which I described ever comes to life (hopefully), then the variable will probably need to be renamed/moved/multiplied again.

Why is that?

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/12/2022 18:04, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
>> 
>> 
>>> syntax-ppss is used in a lot of places. A "free" improvement like the above wouldn't hurt.
>>>
>>>>> So my preference would be to check in the , the var will be there as well.
>>>> I don't insist, but it sounds like the logical thing to do.  So unless
>>>> you feel strongly against that, I'd say let's add that part.  This is,
>>>> after all, an experiment, so let's experiment.
>>> I'm not going to object.
>>>
>>> Doubly so if somebody else wants to finish up the patch and commit it with NEWS and documentation updates. 😉
>> I can finish up the patch if you don't want to 😄
>
> If you don't mind. Thank you. :-)

Ok, will do.  In preparation for that function, we need that variable on
the emacs-29 branch, right?  So here's a patch for that.  What do you
think, Dmitry, Eli and Yuan?  If this gets installed on emacs-29, I can
make the function we discussed on the master branch when that is merged.

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-treesit-text-type-regexp.patch --]
[-- Type: text/x-diff, Size: 10021 bytes --]

From 5edf9bd675b35f97cc0245036ae6ceeb91de1c52 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Thu, 8 Dec 2022 20:06:09 +0100
Subject: [PATCH] Add treesit-text-type-regexp

Sometimes it is good to easily differentiate between textual nodes and
normal code.  One such case is for a filling and reindenting command,
where we fill the textual node if point is on one, and reindent code
otherwise.  This is a part of bug#59662.

* lisp/treesit.el: New variable.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Add
raw_string_literal for C++.
(c-ts-mode, c++-ts-mode): Add regexp for the new variable.
* lisp/progmodes/csharp-mode.el: Add regexp for the new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules): Add
text_block as no-indent.
(java-ts-mode--font-lock-settings): Add text_block font-lock-rule.
(java-ts-mode): Add regexp for the new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* lisp/progmodes/json-ts-mode.el (json-ts-mode--font-lock-settings):
Add comment font-lock-rule.
(json-ts-mode): Add regexp for the new variable, and use the new
comment feature.
* lisp/progmodes/python.el (python-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/textmodes/css-mode.el (css-ts-mode): Add regexp for the new
variable.
---
 lisp/progmodes/c-ts-mode.el          | 12 +++++++++++-
 lisp/progmodes/csharp-mode.el        |  5 +++++
 lisp/progmodes/java-ts-mode.el       |  9 ++++++++-
 lisp/progmodes/js.el                 |  5 +++++
 lisp/progmodes/json-ts-mode.el       |  8 +++++++-
 lisp/progmodes/python.el             |  2 ++
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  4 ++++
 lisp/textmodes/css-mode.el           |  2 ++
 lisp/treesit.el                      |  8 ++++++++
 10 files changed, 56 insertions(+), 3 deletions(-)

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 0b17541a0a..10aec36c2f 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -252,7 +252,9 @@ c-ts-mode--font-lock-settings
    :language mode
    :feature 'string
    `((string_literal) @font-lock-string-face
-     (system_lib_string) @font-lock-string-face)
+     (system_lib_string) @font-lock-string-face
+     ,@(when (eq mode 'cpp)
+         '((raw_string_literal) @font-lock-string-face)))
 
    :language mode
    :feature 'literal
@@ -592,6 +594,10 @@ c-ts-mode
   (setq-local comment-start-skip (rx (or (seq "/" (+ "/"))
                                          (seq "/" (+ "*")))
                                      (* (syntax whitespace))))
+
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment")))
+
   (setq-local comment-end-skip
               (rx (* (syntax whitespace))
                   (group (or (syntax comment-end)
@@ -628,6 +634,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 82e3bc0d54..f83ab16caa 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -912,6 +912,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index bda4ac86c8..30c920be7f 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -73,6 +73,7 @@ java-ts-mode--indent-rules
      ((node-is "]") parent-bol 0)
      ((and (parent-is "comment") comment-end) comment-start -1)
      ((parent-is "comment") comment-start-skip 0)
+     ((parent-is "text_block") no-indent)
      ((parent-is "class_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "interface_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "constructor_body") parent-bol java-ts-mode-indent-offset)
@@ -162,7 +163,8 @@ java-ts-mode--font-lock-settings
    :language 'java
    :override t
    :feature 'string
-   `((string_literal) @font-lock-string-face)
+   `((string_literal) @font-lock-string-face
+     (text_block) @font-lock-string-face)
    :language 'java
    :override t
    :feature 'literal
@@ -313,6 +315,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..d0b243a22f 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
diff --git a/lisp/progmodes/json-ts-mode.el b/lisp/progmodes/json-ts-mode.el
index f3bd93923e..d655821ec9 100644
--- a/lisp/progmodes/json-ts-mode.el
+++ b/lisp/progmodes/json-ts-mode.el
@@ -75,6 +75,9 @@ json-ts--indent-rules
 
 (defvar json-ts-mode--font-lock-settings
   (treesit-font-lock-rules
+   :language 'json
+   :feature 'comment
+   '((comment) @font-lock-comment-face)
    :language 'json
    :feature 'bracket
    '((["[" "]" "{" "}"]) @font-lock-bracket-face)
@@ -159,10 +162,13 @@ json-ts-mode
   (setq-local treesit-defun-type-regexp
               (rx (or "pair" "object")))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment")))
+
   ;; Font-lock.
   (setq-local treesit-font-lock-settings json-ts-mode--font-lock-settings)
   (setq-local treesit-font-lock-feature-list
-              '((constant number pair string)
+              '((comment constant number pair string)
                 (escape-sequence)
                 (bracket delimiter error)))
 
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index ebee703499..f678e10fdf 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -6631,6 +6631,8 @@ python-ts-mode
                 #'python-imenu-treesit-create-index)
     (setq-local treesit-defun-type-regexp (rx (or "function" "class")
                                               "_definition"))
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment")))
     (treesit-major-mode-setup)
 
   (when python-indent-guess-indent-offset
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index a56568ae78..e502c6d249 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -329,6 +329,10 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/textmodes/css-mode.el b/lisp/textmodes/css-mode.el
index 822097a86d..eb4fbc887c 100644
--- a/lisp/textmodes/css-mode.el
+++ b/lisp/textmodes/css-mode.el
@@ -1835,6 +1835,8 @@ css-ts-mode
     (treesit-parser-create 'css)
     (setq-local treesit-simple-indent-rules css--treesit-indent-rules)
     (setq-local treesit-defun-type-regexp "rule_set")
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment")))
     (setq-local treesit-font-lock-settings css--treesit-settings)
     (setq-local treesit-font-lock-feature-list
                 '((selector comment query keyword)
diff --git a/lisp/treesit.el b/lisp/treesit.el
index 85154d0d1c..cc94b460c7 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1639,6 +1639,14 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp nil
+  "A regexp that matches the node type of textual nodes.
+
+A textual is a node that is not normal code, such as comments and
+multiline string literals.  For example, \"(line|block)_comment\"
+in the case of a comment, or \"verbatim_string_literal\" in the
+case of a string.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Sometimes it is good to easily differentiate between textual nodes and
> normal code.  One such case is for a filling and reindenting command,
> where we fill the textual node if point is on one, and reindent code
> otherwise.  This is a part of bug#59662.

I wonder if we shouldn't have something slightly more general, such as
a function which returns nil for "normal code" nodes and non-nil for
"textual" nodes, in which case it could additionally return info such as
whether it's rather like a comment or rather like a string or maybe some
information about how to insert/remove a linebreak from within it (in
case they need to be escaped somehow, or for line-comments).


        Stefan

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: larsi@gnus.org, 59662@debbugs.gnu.org, monnier@iro.umontreal.ca, Yuan Fu
>  <casouri@gmail.com>
> Date: Thu, 08 Dec 2022 20:15:37 +0100
> 
> Ok, will do.  In preparation for that function, we need that variable on
> the emacs-29 branch, right?

Why?  Why cannot we introduce the variable only on master?

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 8 December 2022 21:12:50 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: larsi@gnus.org, 59662@debbugs.gnu.org, monnier@iro.umontreal.ca, Yuan Fu
>>  <casouri@gmail.com>
>> Date: Thu, 08 Dec 2022 20:15:37 +0100
>> 
>> Ok, will do.  In preparation for that function, we need that variable on
>> the emacs-29 branch, right?
>
>Why?  Why cannot we introduce the variable only on master?

Mostly because I want to use it for other stuff, and for merge conflict reasons, but you decide!

I can revise the patch to go on master and introduce the function there.

By the way, what is the deciding factor for what goes where wrt treesit going forward?

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 18:33, Theodor Thornhill wrote:
> 
> 
> On 8 December 2022 17:06:26 CET, Dmitry Gutov <dgutov@yandex.ru> wrote:
>> On 08/12/2022 16:58, Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors wrote:
>>> Dmitry Gutov<dgutov@yandex.ru>  writes:
>>>
>>>> On 08/12/2022 13:09, Theodor Thornhill wrote:
>>>>> What about this?  I changed it to make a point that we should fill if
>>>>> inside of a string or comment, and indent if inside a code block.  We
>>>>> one such variable for each type, or in both as suggested below.
>>>> Could you rephrase maybe? I'm not seeing the difference.
>>> In the first example I made the variable specific to comments.  But
>>> filling should work on everything that's considered text, IIUC.  So if
>>> the variable refers to both the regex needs to accommodate strings and
>>> comments.  That will in turn make it less useful if we want to find
>>> comments specifically, or strings specifically. so we could have both
>>> 'treesit-comment-regexp' and 'treesit-string-regexp' and match on '(or
>>> treesit-comment-regexp treesit-string-regexp)' in this particular case.
>>>
>>> Not really sure if that made more sense.
>>
>> Ah. So that was about the name of the variable. Looks good.
>>
> 
> Yeah, and thus what nodes it should capture
> 
> 
>> Note that if the improvement which I described ever comes to life (hopefully), then the variable will probably need to be renamed/moved/multiplied again.
> 
> Why is that?

Suppose we have a thingy called tree-sitter-parse-state, or tsps in short.

As a general purpose thigy, it would have the means to check whether the 
state corresponds to a string, or to a comment, at least. Maybe it would 
be able to signal different kinds of strings and comments as well, but 
let's put that aside for now.

For clients to get that info, it can have a general function that would 
return the current node (or maybe a derivative of it) -- call it 
tree-sitter-parse-state, and helpers like tsps-string-p and 
tsps-comment-p, as well as, possibly, tsps-comment-or-string-start. 
Maybe other helpers similar to the fields in the ppss defstruct.

And it seems to me that to implement tsps-string-p and tsps-comment-p 
you will need two different constants like that. Then, in turn, 
prog-fill-or-reindent-defun could be implemented in terms of 
tsps-string-p and tsps-comment-p. It could also call 
tsps-comment-or-string[-start], if such helper will be defined, which 
will internally be implemented either in terms of tsps-*-p functions, or 
use the two corresponding new constants anyway.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 08/12/2022 21:15, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Ok, will do.  In preparation for that function, we need that variable on
> the emacs-29 branch, right?

If the variable will be soon substituted with something else, it seems 
unwise to define it on the release branch. But then again, if we declare 
the whole tree-sitter feature as experimental, maybe it'll be okay.

Depends on what else you want to do with it in emacs-29.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 8 December 2022 23:05:37 CET, Dmitry Gutov <dgutov@yandex.ru> wrote:
>On 08/12/2022 21:15, Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors wrote:
>> Ok, will do.  In preparation for that function, we need that variable on
>> the emacs-29 branch, right?
>
>If the variable will be soon substituted with something else, it seems unwise to define it on the release branch. But then again, if we declare the whole tree-sitter feature as experimental, maybe it'll be okay.
>
>Depends on what else you want to do with it in emacs-29.

I was thinking in line of what you wrote in your other mail, but that might be too big to go to emacs-29.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 00:10, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> 
> On 8 December 2022 23:05:37 CET, Dmitry Gutov<dgutov@yandex.ru>  wrote:
>> On 08/12/2022 21:15, Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors wrote:
>>> Ok, will do.  In preparation for that function, we need that variable on
>>> the emacs-29 branch, right?
>> If the variable will be soon substituted with something else, it seems unwise to define it on the release branch. But then again, if we declare the whole tree-sitter feature as experimental, maybe it'll be okay.
>>
>> Depends on what else you want to do with it in emacs-29.
> I was thinking in line of what you wrote in your other mail, but that might be too big to go to emacs-29.

My gut feeling is it will require some time to stabilize, grow the 
required features and whatnot, with associated churn. Then it's better 
to put into master.

But I can just as well be wrong on this subject, and being able to take 
advantage of this feature will be pretty nice.

Speaking of improvements for release, though, have you thought about the 
other idea, the "faster syntax-ppss"? It shouldn't require changes to 
public-facing infrastructure (or very little; like un-obsoleting 
syntax-begin-function). Though whether it will require new constants or 
not, I cannot say.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Thu, 08 Dec 2022 22:21:35 +0100
> From: Theodor Thornhill <theo@thornhill.no>
> CC: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>  monnier@iro.umontreal.ca, casouri@gmail.com
> 
> >> Ok, will do.  In preparation for that function, we need that variable on
> >> the emacs-29 branch, right?
> >
> >Why?  Why cannot we introduce the variable only on master?
> 
> Mostly because I want to use it for other stuff, and for merge conflict reasons, but you decide!
> 
> I can revise the patch to go on master and introduce the function there.

Yes, please do that on master.

> By the way, what is the deciding factor for what goes where wrt treesit going forward?

In general, anything that doesn't need to support the minimal treesit
modes we have now on emacs-29 should go to master.  That includes any
new features, even if they are supported by treesit.  IOW, mot
everything related to tree-sitter should automatically go to the
release branch.

And when in doubt, ask, as always.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 08 Dec 2022 22:21:35 +0100
>> From: Theodor Thornhill <theo@thornhill.no>
>> CC: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>>  monnier@iro.umontreal.ca, casouri@gmail.com
>> 
>> >> Ok, will do.  In preparation for that function, we need that variable on
>> >> the emacs-29 branch, right?
>> >
>> >Why?  Why cannot we introduce the variable only on master?
>> 
>> Mostly because I want to use it for other stuff, and for merge conflict reasons, but you decide!
>> 
>> I can revise the patch to go on master and introduce the function there.
>
> Yes, please do that on master.
>
>> By the way, what is the deciding factor for what goes where wrt treesit going forward?
>
> In general, anything that doesn't need to support the minimal treesit
> modes we have now on emacs-29 should go to master.  That includes any
> new features, even if they are supported by treesit.  IOW, mot
> everything related to tree-sitter should automatically go to the
> release branch.
>
> And when in doubt, ask, as always.

Ok!  Here's the patch.  What do you all think?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 15906 bytes --]

From 6ee9c0caa6677b1047f07235682f948e123f740e Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Thu, 8 Dec 2022 20:06:09 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Add
raw_string_literal for C++.
(c-ts-mode, c++-ts-mode): Add regexp for the new variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules): Add
text_block as no-indent.
(java-ts-mode--font-lock-settings): Add text_block font-lock-rule.
(java-ts-mode): Add regexp for the new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* lisp/progmodes/json-ts-mode.el (json-ts-mode--font-lock-settings):
Add comment font-lock-rule.
(json-ts-mode): Add regexp for the new variable, and use the new
comment feature.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/python.el (python-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
(typescript-ts-mode--indent-rules): Add class_body.
* lisp/textmodes/css-mode.el (css-ts-mode): Add regexp for the new
variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              |  6 +++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          | 12 +++++++++-
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       | 19 +++++++++++++--
 lisp/progmodes/js.el                 |  6 +++++
 lisp/progmodes/json-ts-mode.el       |  8 ++++++-
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/python.el             |  2 ++
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  6 +++++
 lisp/textmodes/css-mode.el           |  2 ++
 lisp/treesit.el                      |  9 +++++++
 13 files changed, 115 insertions(+), 6 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..f9a15e819b 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,12 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+Major modes that derive from @code{prog-mode} can reindent all the
+lines within a defun at point using the command @kbd{M-q}
+(@code{prog-mode-fill-reindent-defun}).  The bounds of of a defun is
+decided by @code{beginning-of-defun-function} and
+@code{end-of-defun-function}.
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..db1ed2a474 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command uses treesit and syntax-ppss to either fill a textual
+element in a defun, such as a doc-string or a comment.  If point is
+inside normal code, it (re)indents the surrounding defun.  Bound by
+default in 'prog-mode-map' to 'M-q'.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..f0728f6fda 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -248,7 +248,9 @@ c-ts-mode--font-lock-settings
    :language mode
    :feature 'string
    `((string_literal) @font-lock-string-face
-     (system_lib_string) @font-lock-string-face)
+     (system_lib_string) @font-lock-string-face
+     ,@(when (eq mode 'cpp)
+         '((raw_string_literal) @font-lock-string-face)))
 
    :language mode
    :feature 'literal
@@ -567,6 +569,10 @@ c-ts-mode
   (setq-local comment-start-skip (rx (or (seq "/" (+ "/"))
                                          (seq "/" (+ "*")))
                                      (* (syntax whitespace))))
+
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment")))
+
   (setq-local comment-end-skip
               (rx (* (syntax whitespace))
                   (group (or (syntax comment-end)
@@ -603,6 +609,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..f29445195a 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -68,6 +68,7 @@ java-ts-mode--indent-rules
      ((node-is "]") parent-bol 0)
      ((and (parent-is "comment") comment-end) comment-start -1)
      ((parent-is "comment") comment-start-skip 0)
+     ((parent-is "text_block") no-indent)
      ((parent-is "class_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "interface_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "constructor_body") parent-bol java-ts-mode-indent-offset)
@@ -157,7 +158,8 @@ java-ts-mode--font-lock-settings
    :language 'java
    :override t
    :feature 'string
-   `((string_literal) @font-lock-string-face)
+   `((string_literal) @font-lock-string-face
+     (text_block) @font-lock-string-face)
    :language 'java
    :override t
    :feature 'literal
@@ -308,6 +310,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
@@ -316,7 +323,15 @@ java-ts-mode
               (append "{}():;," electric-indent-chars))
 
   ;; Navigation.
-  (setq-local treesit-defun-type-regexp "declaration")
+  (setq-local treesit-defun-type-regexp
+              (regexp-opt '("method_declaration"
+                            "class_declaration"
+                            "record_declaration"
+                            "interface_declaration"
+                            "enum_declaration"
+                            "import_declaration"
+                            "package_declaration"
+                            "module_declaration")))
 
   ;; Font-lock.
   (setq-local treesit-font-lock-settings java-ts-mode--font-lock-settings)
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..02cc558366 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
@@ -3868,6 +3873,7 @@ js-ts-mode
     ;; Indent.
     (setq-local treesit-simple-indent-rules js--treesit-indent-rules)
     ;; Navigation.
+    (setq-local treesit-defun-prefer-top-level t)
     (setq-local treesit-defun-type-regexp
                 (rx (or "class_declaration"
                         "method_definition"
diff --git a/lisp/progmodes/json-ts-mode.el b/lisp/progmodes/json-ts-mode.el
index 0a0113d1d8..441e25c287 100644
--- a/lisp/progmodes/json-ts-mode.el
+++ b/lisp/progmodes/json-ts-mode.el
@@ -72,6 +72,9 @@ json-ts--indent-rules
 
 (defvar json-ts-mode--font-lock-settings
   (treesit-font-lock-rules
+   :language 'json
+   :feature 'comment
+   '((comment) @font-lock-comment-face)
    :language 'json
    :feature 'bracket
    '((["[" "]" "{" "}"]) @font-lock-bracket-face)
@@ -156,10 +159,13 @@ json-ts-mode
   (setq-local treesit-defun-type-regexp
               (rx (or "pair" "object")))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment")))
+
   ;; Font-lock.
   (setq-local treesit-font-lock-settings json-ts-mode--font-lock-settings)
   (setq-local treesit-font-lock-feature-list
-              '((constant number pair string)
+              '((comment constant number pair string)
                 (escape-sequence)
                 (bracket delimiter error)))
 
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..c29efa3a21 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index ebee703499..f678e10fdf 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -6631,6 +6631,8 @@ python-ts-mode
                 #'python-imenu-treesit-create-index)
     (setq-local treesit-defun-type-regexp (rx (or "function" "class")
                                               "_definition"))
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment")))
     (treesit-major-mode-setup)
 
   (when python-indent-guess-indent-offset
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..74462bd232 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -74,6 +74,7 @@ typescript-ts-mode--indent-rules
      ((parent-is "member_expression") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "named_imports") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "statement_block") parent-bol typescript-ts-mode-indent-offset)
+     ((parent-is "class_body") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "type_arguments") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "variable_declarator") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "arguments") parent-bol typescript-ts-mode-indent-offset)
@@ -323,6 +324,11 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-defun-prefer-top-level t)
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/textmodes/css-mode.el b/lisp/textmodes/css-mode.el
index 8a66986dc6..f5b7c3ce3d 100644
--- a/lisp/textmodes/css-mode.el
+++ b/lisp/textmodes/css-mode.el
@@ -1834,6 +1834,8 @@ css-ts-mode
     (treesit-parser-create 'css)
     (setq-local treesit-simple-indent-rules css--treesit-indent-rules)
     (setq-local treesit-defun-type-regexp "rule_set")
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment")))
     (setq-local treesit-font-lock-settings css--treesit-settings)
     (setq-local treesit-font-lock-feature-list
                 '((selector comment query keyword)
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..11fb699180 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp nil
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 15:24, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Ok!  Here's the patch.  What do you all think?

- The NEWS entry probably doesn't need to mention syntax-ppss or 
tree-sitter. Those are just implementation details.

- treesit-text-type-regexp could have a default value. (regexp-opt 
'("comment")) seems popular enough.

Looks good overall.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 501 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 09/12/2022 15:24, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
>> Ok!  Here's the patch.  What do you all think?
>
> - The NEWS entry probably doesn't need to mention syntax-ppss or 
> tree-sitter. Those are just implementation details.
>
> - treesit-text-type-regexp could have a default value. (regexp-opt 
> '("comment")) seems popular enough.
>

Something like this?

> Looks good overall.

Thanks!


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 13828 bytes --]

From eb5d05795129ddd0a9aa3f8959e15cb54a9030bc Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Thu, 8 Dec 2022 20:06:09 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Add
raw_string_literal for C++.
(c++-ts-mode): Add regexp for the new variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules): Add
text_block as no-indent.
(java-ts-mode--font-lock-settings): Add text_block font-lock-rule.
(java-ts-mode): Add regexp for the new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* lisp/progmodes/json-ts-mode.el (json-ts-mode--font-lock-settings):
Add comment font-lock-rule.
(json-ts-mode): Use the new comment feature.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
(typescript-ts-mode--indent-rules): Add class_body.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              |  6 +++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  8 ++++++-
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       | 19 +++++++++++++--
 lisp/progmodes/js.el                 |  6 +++++
 lisp/progmodes/json-ts-mode.el       |  5 +++-
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  6 +++++
 lisp/treesit.el                      |  9 +++++++
 11 files changed, 104 insertions(+), 6 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..f9a15e819b 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,12 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+Major modes that derive from @code{prog-mode} can reindent all the
+lines within a defun at point using the command @kbd{M-q}
+(@code{prog-mode-fill-reindent-defun}).  The bounds of of a defun is
+decided by @code{beginning-of-defun-function} and
+@code{end-of-defun-function}.
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..6216fd3681 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a textual element in a defun, such as a
+doc-string, or a comment.  If point is inside normal code, it
+(re)indents the surrounding defun.  Bound by default in
+'prog-mode-map' to 'M-q'.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..f7285e8dd6 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -248,7 +248,9 @@ c-ts-mode--font-lock-settings
    :language mode
    :feature 'string
    `((string_literal) @font-lock-string-face
-     (system_lib_string) @font-lock-string-face)
+     (system_lib_string) @font-lock-string-face
+     ,@(when (eq mode 'cpp)
+         '((raw_string_literal) @font-lock-string-face)))
 
    :language mode
    :feature 'literal
@@ -603,6 +605,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..f29445195a 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -68,6 +68,7 @@ java-ts-mode--indent-rules
      ((node-is "]") parent-bol 0)
      ((and (parent-is "comment") comment-end) comment-start -1)
      ((parent-is "comment") comment-start-skip 0)
+     ((parent-is "text_block") no-indent)
      ((parent-is "class_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "interface_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "constructor_body") parent-bol java-ts-mode-indent-offset)
@@ -157,7 +158,8 @@ java-ts-mode--font-lock-settings
    :language 'java
    :override t
    :feature 'string
-   `((string_literal) @font-lock-string-face)
+   `((string_literal) @font-lock-string-face
+     (text_block) @font-lock-string-face)
    :language 'java
    :override t
    :feature 'literal
@@ -308,6 +310,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
@@ -316,7 +323,15 @@ java-ts-mode
               (append "{}():;," electric-indent-chars))
 
   ;; Navigation.
-  (setq-local treesit-defun-type-regexp "declaration")
+  (setq-local treesit-defun-type-regexp
+              (regexp-opt '("method_declaration"
+                            "class_declaration"
+                            "record_declaration"
+                            "interface_declaration"
+                            "enum_declaration"
+                            "import_declaration"
+                            "package_declaration"
+                            "module_declaration")))
 
   ;; Font-lock.
   (setq-local treesit-font-lock-settings java-ts-mode--font-lock-settings)
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..02cc558366 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
@@ -3868,6 +3873,7 @@ js-ts-mode
     ;; Indent.
     (setq-local treesit-simple-indent-rules js--treesit-indent-rules)
     ;; Navigation.
+    (setq-local treesit-defun-prefer-top-level t)
     (setq-local treesit-defun-type-regexp
                 (rx (or "class_declaration"
                         "method_definition"
diff --git a/lisp/progmodes/json-ts-mode.el b/lisp/progmodes/json-ts-mode.el
index 0a0113d1d8..736ff7f3cd 100644
--- a/lisp/progmodes/json-ts-mode.el
+++ b/lisp/progmodes/json-ts-mode.el
@@ -72,6 +72,9 @@ json-ts--indent-rules
 
 (defvar json-ts-mode--font-lock-settings
   (treesit-font-lock-rules
+   :language 'json
+   :feature 'comment
+   '((comment) @font-lock-comment-face)
    :language 'json
    :feature 'bracket
    '((["[" "]" "{" "}"]) @font-lock-bracket-face)
@@ -159,7 +162,7 @@ json-ts-mode
   ;; Font-lock.
   (setq-local treesit-font-lock-settings json-ts-mode--font-lock-settings)
   (setq-local treesit-font-lock-feature-list
-              '((constant number pair string)
+              '((comment constant number pair string)
                 (escape-sequence)
                 (bracket delimiter error)))
 
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..c29efa3a21 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..74462bd232 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -74,6 +74,7 @@ typescript-ts-mode--indent-rules
      ((parent-is "member_expression") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "named_imports") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "statement_block") parent-bol typescript-ts-mode-indent-offset)
+     ((parent-is "class_body") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "type_arguments") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "variable_declarator") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "arguments") parent-bol typescript-ts-mode-indent-offset)
@@ -323,6 +324,11 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-defun-prefer-top-level t)
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..32ec9ce92f 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp (regexp-opt '("comment"))
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 16:46, Theodor Thornhill wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> On 09/12/2022 15:24, Theodor Thornhill via Bug reports for GNU Emacs,
>> the Swiss army knife of text editors wrote:
>>> Ok!  Here's the patch.  What do you all think?
>>
>> - The NEWS entry probably doesn't need to mention syntax-ppss or
>> tree-sitter. Those are just implementation details.
>>
>> - treesit-text-type-regexp could have a default value. (regexp-opt
>> '("comment")) seems popular enough.
>>
> 
> Something like this?

Yes. But... :-)

I guess you can replace (regexp-opt '("comment")) with just "comment"? 
Both regexps are equivalent, and neither uses anchoring (like \` and 
\'). Should they have anchoring?

The programs.texi addition should probably mention refilling too.

And I see some additional changes in this patch, including ones that 
improve defun navigation, font-lock and indentation, in particular in 
java-ts-mode (but also json and typescript). These changes should 
probably go to emacs-29.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 09/12/2022 16:46, Theodor Thornhill wrote:
>> Dmitry Gutov <dgutov@yandex.ru> writes:
>> 
>>> On 09/12/2022 15:24, Theodor Thornhill via Bug reports for GNU Emacs,
>>> the Swiss army knife of text editors wrote:
>>>> Ok!  Here's the patch.  What do you all think?
>>>
>>> - The NEWS entry probably doesn't need to mention syntax-ppss or
>>> tree-sitter. Those are just implementation details.
>>>
>>> - treesit-text-type-regexp could have a default value. (regexp-opt
>>> '("comment")) seems popular enough.
>>>
>> 
>> Something like this?
>
> Yes. But... :-)
>
> I guess you can replace (regexp-opt '("comment")) with just "comment"? 
> Both regexps are equivalent, and neither uses anchoring (like \` and 
> \'). Should they have anchoring?
>

Hehe, I did that first, but I trusted you knew some secret about that
expression.  No, we get the names, so I think it's good.

> The programs.texi addition should probably mention refilling too.
>

Good point.  Will fix that.

> And I see some additional changes in this patch, including ones that 
> improve defun navigation, font-lock and indentation, in particular in 
> java-ts-mode (but also json and typescript). These changes should 
> probably go to emacs-29.

Yeah, I agree.  Some of them are needed for this function to work
properly, but the patches I've submitted for them have been lingering
for a while.

See below patch - I guess we can decide on that last remark later :-)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 13887 bytes --]

From fd3fd93b5c7f1340b6cc5d94a5640a999010536d Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Thu, 8 Dec 2022 20:06:09 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Add
raw_string_literal for C++.
(c++-ts-mode): Add regexp for the new variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules): Add
text_block as no-indent.
(java-ts-mode--font-lock-settings): Add text_block font-lock-rule.
(java-ts-mode): Add regexp for the new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* lisp/progmodes/json-ts-mode.el (json-ts-mode--font-lock-settings):
Add comment font-lock-rule.
(json-ts-mode): Use the new comment feature.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
(typescript-ts-mode--indent-rules): Add class_body.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              |  7 ++++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  8 ++++++-
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       | 19 +++++++++++++--
 lisp/progmodes/js.el                 |  6 +++++
 lisp/progmodes/json-ts-mode.el       |  5 +++-
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  6 +++++
 lisp/treesit.el                      |  9 +++++++
 11 files changed, 105 insertions(+), 6 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..5a3d56a228 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,13 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+Major modes that derive from @code{prog-mode} can reindent or refill
+all the lines within a defun at point using the command @kbd{M-q}
+(@code{prog-mode-fill-reindent-defun}).  The bounds of of a defun is
+decided by @code{beginning-of-defun-function} and
+@code{end-of-defun-function}.  If point is inside a comment or a
+doc-string, refill instead.
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..6216fd3681 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a textual element in a defun, such as a
+doc-string, or a comment.  If point is inside normal code, it
+(re)indents the surrounding defun.  Bound by default in
+'prog-mode-map' to 'M-q'.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..f7285e8dd6 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -248,7 +248,9 @@ c-ts-mode--font-lock-settings
    :language mode
    :feature 'string
    `((string_literal) @font-lock-string-face
-     (system_lib_string) @font-lock-string-face)
+     (system_lib_string) @font-lock-string-face
+     ,@(when (eq mode 'cpp)
+         '((raw_string_literal) @font-lock-string-face)))
 
    :language mode
    :feature 'literal
@@ -603,6 +605,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..f29445195a 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -68,6 +68,7 @@ java-ts-mode--indent-rules
      ((node-is "]") parent-bol 0)
      ((and (parent-is "comment") comment-end) comment-start -1)
      ((parent-is "comment") comment-start-skip 0)
+     ((parent-is "text_block") no-indent)
      ((parent-is "class_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "interface_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "constructor_body") parent-bol java-ts-mode-indent-offset)
@@ -157,7 +158,8 @@ java-ts-mode--font-lock-settings
    :language 'java
    :override t
    :feature 'string
-   `((string_literal) @font-lock-string-face)
+   `((string_literal) @font-lock-string-face
+     (text_block) @font-lock-string-face)
    :language 'java
    :override t
    :feature 'literal
@@ -308,6 +310,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
@@ -316,7 +323,15 @@ java-ts-mode
               (append "{}():;," electric-indent-chars))
 
   ;; Navigation.
-  (setq-local treesit-defun-type-regexp "declaration")
+  (setq-local treesit-defun-type-regexp
+              (regexp-opt '("method_declaration"
+                            "class_declaration"
+                            "record_declaration"
+                            "interface_declaration"
+                            "enum_declaration"
+                            "import_declaration"
+                            "package_declaration"
+                            "module_declaration")))
 
   ;; Font-lock.
   (setq-local treesit-font-lock-settings java-ts-mode--font-lock-settings)
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..02cc558366 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
@@ -3868,6 +3873,7 @@ js-ts-mode
     ;; Indent.
     (setq-local treesit-simple-indent-rules js--treesit-indent-rules)
     ;; Navigation.
+    (setq-local treesit-defun-prefer-top-level t)
     (setq-local treesit-defun-type-regexp
                 (rx (or "class_declaration"
                         "method_definition"
diff --git a/lisp/progmodes/json-ts-mode.el b/lisp/progmodes/json-ts-mode.el
index 0a0113d1d8..736ff7f3cd 100644
--- a/lisp/progmodes/json-ts-mode.el
+++ b/lisp/progmodes/json-ts-mode.el
@@ -72,6 +72,9 @@ json-ts--indent-rules
 
 (defvar json-ts-mode--font-lock-settings
   (treesit-font-lock-rules
+   :language 'json
+   :feature 'comment
+   '((comment) @font-lock-comment-face)
    :language 'json
    :feature 'bracket
    '((["[" "]" "{" "}"]) @font-lock-bracket-face)
@@ -159,7 +162,7 @@ json-ts-mode
   ;; Font-lock.
   (setq-local treesit-font-lock-settings json-ts-mode--font-lock-settings)
   (setq-local treesit-font-lock-feature-list
-              '((constant number pair string)
+              '((comment constant number pair string)
                 (escape-sequence)
                 (bracket delimiter error)))
 
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..c29efa3a21 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..74462bd232 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -74,6 +74,7 @@ typescript-ts-mode--indent-rules
      ((parent-is "member_expression") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "named_imports") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "statement_block") parent-bol typescript-ts-mode-indent-offset)
+     ((parent-is "class_body") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "type_arguments") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "variable_declarator") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "arguments") parent-bol typescript-ts-mode-indent-offset)
@@ -323,6 +324,11 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-defun-prefer-top-level t)
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..a7b4ae1591 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp "comment"
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 17:13, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
>>> Something like this?
>>
>> Yes. But... :-)
>>
>> I guess you can replace (regexp-opt '("comment")) with just "comment"?
>> Both regexps are equivalent, and neither uses anchoring (like \` and
>> \'). Should they have anchoring?
>>
> 
> Hehe, I did that first, but I trusted you knew some secret about that
> expression.  No, we get the names, so I think it's good.

Shouldn't trust me that much ;-)

It could be a problem if some grammar somewhere will have a node type 
called "not_comment", and the regexp will still match it. Or 
"pseudo_string", etc.

Not sure how realistic that scenario is, up to you. The authors of such 
major modes could opt for a stricted regexp individually, too.

>> The programs.texi addition should probably mention refilling too.
>>
> 
> Good point.  Will fix that.

Still not ideal: it says "refill all the lines within a defun", but the 
refilling step only touches the paragraph around point.

Also about NEWS: "textual element" is a little value; it really refills 
a single paragraph. That distinction is meaningful when a string or a 
comment are long, containing several paragraphs inside.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

Sorry:

On 09/12/2022 17:27, Dmitry Gutov wrote:
> "textual element" is a little value;

                                 ^ vague

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1485 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 09/12/2022 17:13, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
>>>> Something like this?
>>>
>>> Yes. But... :-)
>>>
>>> I guess you can replace (regexp-opt '("comment")) with just "comment"?
>>> Both regexps are equivalent, and neither uses anchoring (like \` and
>>> \'). Should they have anchoring?
>>>
>> 
>> Hehe, I did that first, but I trusted you knew some secret about that
>> expression.  No, we get the names, so I think it's good.
>
> Shouldn't trust me that much ;-)
>

:-)

> It could be a problem if some grammar somewhere will have a node type 
> called "not_comment", and the regexp will still match it. Or 
> "pseudo_string", etc.
>
> Not sure how realistic that scenario is, up to you. The authors of such 
> major modes could opt for a stricted regexp individually, too.
>

Right.  Yeah, I don't think it'll be a problem, but better to be on the
safe side.  Tweaked the regexp a little

>>> The programs.texi addition should probably mention refilling too.
>>>
>> 
>> Good point.  Will fix that.
>
> Still not ideal: it says "refill all the lines within a defun", but the 
> refilling step only touches the paragraph around point.
>
> Also about NEWS: "textual element" is a little value; it really refills 
> a single paragraph. That distinction is meaningful when a string or a 
> comment are long, containing several paragraphs inside.

Something like this?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 13919 bytes --]

From 8e8e045423f6aba96d660f30a7a4e96073d2ad75 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Thu, 8 Dec 2022 20:06:09 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Add
raw_string_literal for C++.
(c++-ts-mode): Add regexp for the new variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules): Add
text_block as no-indent.
(java-ts-mode--font-lock-settings): Add text_block font-lock-rule.
(java-ts-mode): Add regexp for the new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* lisp/progmodes/json-ts-mode.el (json-ts-mode--font-lock-settings):
Add comment font-lock-rule.
(json-ts-mode): Use the new comment feature.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
(typescript-ts-mode--indent-rules): Add class_body.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              |  8 +++++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  8 ++++++-
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       | 19 +++++++++++++--
 lisp/progmodes/js.el                 |  6 +++++
 lisp/progmodes/json-ts-mode.el       |  5 +++-
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  6 +++++
 lisp/treesit.el                      |  9 +++++++
 11 files changed, 106 insertions(+), 6 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..cc06a003e2 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,14 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+Major modes that derive from @code{prog-mode} can reindent all the
+lines within a defun around point using
+@code{prog-mode-fill-reindent-defun}.  The bounds of of a defun is
+decided by @code{beginning-of-defun-function} and
+@code{end-of-defun-function}.  If point is inside a comment or a
+doc-string, refill that paragraph instead, using
+@code{fill-paragraph-function}.
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..659ee39faf 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a single paragraph in a defun, such as a
+doc-string, or a comment.  If point is inside normal code, it
+(re)indents the surrounding defun.  Bound by default in
+'prog-mode-map' to 'M-q'.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..f7285e8dd6 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -248,7 +248,9 @@ c-ts-mode--font-lock-settings
    :language mode
    :feature 'string
    `((string_literal) @font-lock-string-face
-     (system_lib_string) @font-lock-string-face)
+     (system_lib_string) @font-lock-string-face
+     ,@(when (eq mode 'cpp)
+         '((raw_string_literal) @font-lock-string-face)))
 
    :language mode
    :feature 'literal
@@ -603,6 +605,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..f29445195a 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -68,6 +68,7 @@ java-ts-mode--indent-rules
      ((node-is "]") parent-bol 0)
      ((and (parent-is "comment") comment-end) comment-start -1)
      ((parent-is "comment") comment-start-skip 0)
+     ((parent-is "text_block") no-indent)
      ((parent-is "class_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "interface_body") parent-bol java-ts-mode-indent-offset)
      ((parent-is "constructor_body") parent-bol java-ts-mode-indent-offset)
@@ -157,7 +158,8 @@ java-ts-mode--font-lock-settings
    :language 'java
    :override t
    :feature 'string
-   `((string_literal) @font-lock-string-face)
+   `((string_literal) @font-lock-string-face
+     (text_block) @font-lock-string-face)
    :language 'java
    :override t
    :feature 'literal
@@ -308,6 +310,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
@@ -316,7 +323,15 @@ java-ts-mode
               (append "{}():;," electric-indent-chars))
 
   ;; Navigation.
-  (setq-local treesit-defun-type-regexp "declaration")
+  (setq-local treesit-defun-type-regexp
+              (regexp-opt '("method_declaration"
+                            "class_declaration"
+                            "record_declaration"
+                            "interface_declaration"
+                            "enum_declaration"
+                            "import_declaration"
+                            "package_declaration"
+                            "module_declaration")))
 
   ;; Font-lock.
   (setq-local treesit-font-lock-settings java-ts-mode--font-lock-settings)
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..02cc558366 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
@@ -3868,6 +3873,7 @@ js-ts-mode
     ;; Indent.
     (setq-local treesit-simple-indent-rules js--treesit-indent-rules)
     ;; Navigation.
+    (setq-local treesit-defun-prefer-top-level t)
     (setq-local treesit-defun-type-regexp
                 (rx (or "class_declaration"
                         "method_definition"
diff --git a/lisp/progmodes/json-ts-mode.el b/lisp/progmodes/json-ts-mode.el
index 0a0113d1d8..736ff7f3cd 100644
--- a/lisp/progmodes/json-ts-mode.el
+++ b/lisp/progmodes/json-ts-mode.el
@@ -72,6 +72,9 @@ json-ts--indent-rules
 
 (defvar json-ts-mode--font-lock-settings
   (treesit-font-lock-rules
+   :language 'json
+   :feature 'comment
+   '((comment) @font-lock-comment-face)
    :language 'json
    :feature 'bracket
    '((["[" "]" "{" "}"]) @font-lock-bracket-face)
@@ -159,7 +162,7 @@ json-ts-mode
   ;; Font-lock.
   (setq-local treesit-font-lock-settings json-ts-mode--font-lock-settings)
   (setq-local treesit-font-lock-feature-list
-              '((constant number pair string)
+              '((comment constant number pair string)
                 (escape-sequence)
                 (bracket delimiter error)))
 
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..c29efa3a21 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..74462bd232 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -74,6 +74,7 @@ typescript-ts-mode--indent-rules
      ((parent-is "member_expression") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "named_imports") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "statement_block") parent-bol typescript-ts-mode-indent-offset)
+     ((parent-is "class_body") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "type_arguments") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "variable_declarator") parent-bol typescript-ts-mode-indent-offset)
      ((parent-is "arguments") parent-bol typescript-ts-mode-indent-offset)
@@ -323,6 +324,11 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-defun-prefer-top-level t)
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..41feac35c2 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp "\\`comment\\'"
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 17:50, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Something like this?

Looking good, thanks!

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 09/12/2022 17:50, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
>> Something like this?
>
> Looking good, thanks!

My pleasure.  I removed the code that should go to emacs-29.  Can you
install if you agree with this patch, Dmitry?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 9809 bytes --]

From 6f5e1c917ce49245ad40f12ed6789e6cb33e51fd Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 9 Dec 2022 20:12:51 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c++-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              |  8 +++++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  4 ++++
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       |  5 ++++
 lisp/progmodes/js.el                 |  5 ++++
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  4 ++++
 lisp/treesit.el                      |  9 +++++++
 10 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..cc06a003e2 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,14 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+Major modes that derive from @code{prog-mode} can reindent all the
+lines within a defun around point using
+@code{prog-mode-fill-reindent-defun}.  The bounds of of a defun is
+decided by @code{beginning-of-defun-function} and
+@code{end-of-defun-function}.  If point is inside a comment or a
+doc-string, refill that paragraph instead, using
+@code{fill-paragraph-function}.
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..659ee39faf 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a single paragraph in a defun, such as a
+doc-string, or a comment.  If point is inside normal code, it
+(re)indents the surrounding defun.  Bound by default in
+'prog-mode-map' to 'M-q'.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..114ffc0b03 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -603,6 +603,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..b1bf965e4f 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -308,6 +308,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..d0b243a22f 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..c29efa3a21 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..59598b4c63 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -323,6 +323,10 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..41feac35c2 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp "\\`comment\\'"
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: larsi@gnus.org, 59662@debbugs.gnu.org, casouri@gmail.com,
>  monnier@iro.umontreal.ca
> Date: Fri, 09 Dec 2022 16:50:25 +0100
> 
> diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
> index ba8475e86a..cc06a003e2 100644
> --- a/doc/emacs/programs.texi
> +++ b/doc/emacs/programs.texi
> @@ -409,6 +409,14 @@ Multi-line Indent
>  @table @kbd
>  @item C-M-q
>  Reindent all the lines within one parenthetical grouping.
> +@item M-q
> +Major modes that derive from @code{prog-mode} can reindent all the
> +lines within a defun around point using
> +@code{prog-mode-fill-reindent-defun}.  The bounds of of a defun is
> +decided by @code{beginning-of-defun-function} and
> +@code{end-of-defun-function}.  If point is inside a comment or a
> +doc-string, refill that paragraph instead, using
> +@code{fill-paragraph-function}.

The 2 variables you mention should have @vindex entries.  Also, please
add a cross-reference to the mode in elisp manual where these two
variables are described.

> +*** New command 'prog-fill-reindent-defun'
> +This command either fills a single paragraph in a defun, such as a
> +doc-string, or a comment.  If point is inside normal code, it
> +(re)indents the surrounding defun.

These two sentences should be one sentence, otherwise that 'either" is
out of place.  Something like

  This command either fills a single paragraph in a defun, such as a
  doc-string, or a comment, or (re)indents the surrounding defun if
  point is not in a comment or a string.

>                                        Bound by default in
> +'prog-mode-map' to 'M-q'.

"It is by default bound to 'M-q' in 'prog-mode' and all its
descendants."

> +(defun prog-fill-reindent-defun (&optional argument)
> +  "Refill paragraph or reindent the definition that the point is on.
> +
> +If the point is in a string, or in a comment, or there is a
> +comment on the current line, fill the paragraph that the point is
> +in or is on the same line.

I don't understand the difference between "point is in a string, or in
a comment" and "there is a comment on the current line".  

> +Otherwise, reindent the definition around or below point."

Likewise this one is confusing.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 21:15, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> On 09/12/2022 17:50, Theodor Thornhill via Bug reports for GNU Emacs,
>> the Swiss army knife of text editors wrote:
>>> Something like this?
>>
>> Looking good, thanks!
> 
> My pleasure.  I removed the code that should go to emacs-29.  Can you
> install if you agree with this patch, Dmitry?

LGTM.

I expected Eli would also take a look, but since it's for master, I 
suppose we don't have to wait for his sign-off.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 22:20, Eli Zaretskii wrote:

>> +(defun prog-fill-reindent-defun (&optional argument)
>> +  "Refill paragraph or reindent the definition that the point is on.
>> +
>> +If the point is in a string, or in a comment, or there is a
>> +comment on the current line, fill the paragraph that the point is
>> +in or is on the same line.
> 
> I don't understand the difference between "point is in a string, or in
> a comment" and "there is a comment on the current line".

The comment may start after point.

>> +Otherwise, reindent the definition around or below point."
> 
> Likewise this one is confusing.

The intent was to describe the behavior that follows 'mark-defun'. Which 
has this in its docstring:

   Put mark at end of this defun, point at beginning.
   The defun marked is the one that contains point or follows point.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 09/12/2022 22:28, Dmitry Gutov wrote:
> I expected Eli would also take a look, but since it's for master, I 
> suppose we don't have to wait for his sign-off.

And scratch that.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2663 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: larsi@gnus.org, 59662@debbugs.gnu.org, casouri@gmail.com,
>>  monnier@iro.umontreal.ca
>> Date: Fri, 09 Dec 2022 16:50:25 +0100
>> 
>> diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
>> index ba8475e86a..cc06a003e2 100644
>> --- a/doc/emacs/programs.texi
>> +++ b/doc/emacs/programs.texi
>> @@ -409,6 +409,14 @@ Multi-line Indent
>>  @table @kbd
>>  @item C-M-q
>>  Reindent all the lines within one parenthetical grouping.
>> +@item M-q
>> +Major modes that derive from @code{prog-mode} can reindent all the
>> +lines within a defun around point using
>> +@code{prog-mode-fill-reindent-defun}.  The bounds of of a defun is
>> +decided by @code{beginning-of-defun-function} and
>> +@code{end-of-defun-function}.  If point is inside a comment or a
>> +doc-string, refill that paragraph instead, using
>> +@code{fill-paragraph-function}.
>
> The 2 variables you mention should have @vindex entries.  Also, please
> add a cross-reference to the mode in elisp manual where these two
> variables are described.
>
>> +*** New command 'prog-fill-reindent-defun'
>> +This command either fills a single paragraph in a defun, such as a
>> +doc-string, or a comment.  If point is inside normal code, it
>> +(re)indents the surrounding defun.
>
> These two sentences should be one sentence, otherwise that 'either" is
> out of place.  Something like
>
>   This command either fills a single paragraph in a defun, such as a
>   doc-string, or a comment, or (re)indents the surrounding defun if
>   point is not in a comment or a string.
>
>>                                        Bound by default in
>> +'prog-mode-map' to 'M-q'.

Something like the patch below?  Forgive me, but my texinfo-fu is
limited.  I'm a little unsure if I xrefed to the correct nodes.  Maybe
it is better to just mention 'beginning-of-defun' rather than the
variable 'beginning-of-defun-function'?

>
> "It is by default bound to 'M-q' in 'prog-mode' and all its
> descendants."
>
>> +(defun prog-fill-reindent-defun (&optional argument)
>> +  "Refill paragraph or reindent the definition that the point is on.
>> +
>> +If the point is in a string, or in a comment, or there is a
>> +comment on the current line, fill the paragraph that the point is
>> +in or is on the same line.
>
> I don't understand the difference between "point is in a string, or in
> a comment" and "there is a comment on the current line".  
>
>> +Otherwise, reindent the definition around or below point."
>
> Likewise this one is confusing.

I'm not sure how to word this differently, but I agree it's a mouthful.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 10064 bytes --]

From 7d2ae8eb9bcfd1409d7ee3c79f1a98663eac9832 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 9 Dec 2022 20:12:51 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c++-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              | 13 +++++++++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  4 ++++
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       |  5 ++++
 lisp/progmodes/js.el                 |  5 ++++
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  4 ++++
 lisp/treesit.el                      |  9 +++++++
 10 files changed, 89 insertions(+), 2 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..d981ef6b49 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,19 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+@vindex beginning-of-defun-function
+@vindex end-of-defun-function
+@vindex fill-paragraph-function
+@findex prog-mode-fill-reindent-defun
+Major modes that derive from @code{prog-mode} can reindent all the
+lines within a defun around point using
+@code{prog-mode-fill-reindent-defun}.  The bounds of a defun is
+decided by @code{beginning-of-defun-function} and
+@code{end-of-defun-function} (@pxref{Moving by Defuns}).  If point is
+inside a comment or a doc-string, refill that paragraph instead, using
+@code{fill-paragraph-function} (@xref{Filling}, for more information
+on @kbd{M-q}.)
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..a5eb7f332d 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a single paragraph in a defun, such as a
+doc-string, or a comment, or (re)indents the surrounding defun if
+point is not in a comment or a string.  It is by default bound to
+'M-q' in 'prog-mode' and all its descendants.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..114ffc0b03 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -603,6 +603,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..b1bf965e4f 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -308,6 +308,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..d0b243a22f 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..c29efa3a21 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..59598b4c63 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -323,6 +323,10 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..41feac35c2 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp "\\`comment\\'"
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1231 bytes --]

Theodor Thornhill <theo@thornhill.no> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Theodor Thornhill <theo@thornhill.no>
>>> Cc: larsi@gnus.org, 59662@debbugs.gnu.org, casouri@gmail.com,
>>>  monnier@iro.umontreal.ca
>>> Date: Fri, 09 Dec 2022 16:50:25 +0100
>>> 
>>> diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
>>> index ba8475e86a..cc06a003e2 100644
>>> --- a/doc/emacs/programs.texi
>>> +++ b/doc/emacs/programs.texi
>>> @@ -409,6 +409,14 @@ Multi-line Indent
>>>  @table @kbd
>>>  @item C-M-q
>>>  Reindent all the lines within one parenthetical grouping.
>>> +@item M-q
>>> +Major modes that derive from @code{prog-mode} can reindent all the
>>> +lines within a defun around point using
>>> +@code{prog-mode-fill-reindent-defun}.  The bounds of of a defun is
>>> +decided by @code{beginning-of-defun-function} and
>>> +@code{end-of-defun-function}.  If point is inside a comment or a
>>> +doc-string, refill that paragraph instead, using
>>> +@code{fill-paragraph-function}.
>>
>> The 2 variables you mention should have @vindex entries.  Also, please
>> add a cross-reference to the mode in elisp manual where these two
>> variables are described.
>>

Is this worded a little better?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 9874 bytes --]

From 5b160b3e45371b35919e29e2152737feca10e8cf Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 9 Dec 2022 20:12:51 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c++-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              |  8 +++++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  4 ++++
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       |  5 ++++
 lisp/progmodes/js.el                 |  5 ++++
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  4 ++++
 lisp/treesit.el                      |  9 +++++++
 10 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..06bff21a48 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,14 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+@findex prog-mode-fill-reindent-defun
+Major modes that derive from @code{prog-mode} can reindent all the
+lines within a defun around point using
+@code{prog-mode-fill-reindent-defun} (@pxref{Defuns}, for more
+information about defuns).  If point is inside a comment or a
+doc-string, refill that paragraph instead, using @code{fill-paragraph}
+(@pxref{Fill Commands}, for more information on @kbd{M-q}.)
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..a5eb7f332d 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a single paragraph in a defun, such as a
+doc-string, or a comment, or (re)indents the surrounding defun if
+point is not in a comment or a string.  It is by default bound to
+'M-q' in 'prog-mode' and all its descendants.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..114ffc0b03 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -603,6 +603,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..b1bf965e4f 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -308,6 +308,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..d0b243a22f 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..c29efa3a21 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill paragraph or reindent the definition that the point is on.
+
+If the point is in a string, or in a comment, or there is a
+comment on the current line, fill the paragraph that the point is
+in or is on the same line.
+
+Otherwise, reindent the definition around or below point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..59598b4c63 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -323,6 +323,10 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..41feac35c2 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp "\\`comment\\'"
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> Date: Fri, 9 Dec 2022 22:30:55 +0200
> Cc: larsi@gnus.org, 59662@debbugs.gnu.org, casouri@gmail.com,
>  monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 09/12/2022 22:20, Eli Zaretskii wrote:
> 
> >> +(defun prog-fill-reindent-defun (&optional argument)
> >> +  "Refill paragraph or reindent the definition that the point is on.
> >> +
> >> +If the point is in a string, or in a comment, or there is a
> >> +comment on the current line, fill the paragraph that the point is
> >> +in or is on the same line.
> > 
> > I don't understand the difference between "point is in a string, or in
> > a comment" and "there is a comment on the current line".
> 
> The comment may start after point.

Only after, not before?

> >> +Otherwise, reindent the definition around or below point."
> > 
> > Likewise this one is confusing.
> 
> The intent was to describe the behavior that follows 'mark-defun'. Which 
> has this in its docstring:
> 
>    Put mark at end of this defun, point at beginning.
>    The defun marked is the one that contains point or follows point.

That one is clear.  Does "below point" means before point (i.e. lower
buffer position) or after point (i.e. lower on the screen)?

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>  casouri@gmail.com, monnier@iro.umontreal.ca
> Date: Fri, 09 Dec 2022 22:09:10 +0100
> 
> Something like the patch below?  Forgive me, but my texinfo-fu is
> limited.

You have nothing to worry in that department.

> I'm a little unsure if I xrefed to the correct nodes.  Maybe
> it is better to just mention 'beginning-of-defun' rather than the
> variable 'beginning-of-defun-function'?

I meant a reference to where beginning-of-defun-function is mentioned,
and that is in the ELisp manual.  You can see examples of how to
reference another manual in many places in doc/emacs/*.texi; seartch
for "elisp".

> >> +(defun prog-fill-reindent-defun (&optional argument)
> >> +  "Refill paragraph or reindent the definition that the point is on.
> >> +
> >> +If the point is in a string, or in a comment, or there is a
> >> +comment on the current line, fill the paragraph that the point is
> >> +in or is on the same line.
> >
> > I don't understand the difference between "point is in a string, or in
> > a comment" and "there is a comment on the current line".  
> >
> >> +Otherwise, reindent the definition around or below point."
> >
> > Likewise this one is confusing.
> 
> I'm not sure how to word this differently, but I agree it's a mouthful.

If you explain to me in plain English what you meant, I can suggest a
better wording.

Thanks.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>  casouri@gmail.com, monnier@iro.umontreal.ca
> Date: Sat, 10 Dec 2022 13:02:09 +0100
> 
> Is this worded a little better?

I think my last message answers this as well.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 10/12/2022 16:04, Eli Zaretskii wrote:
>> Date: Fri, 9 Dec 2022 22:30:55 +0200
>> Cc:larsi@gnus.org,59662@debbugs.gnu.org,casouri@gmail.com,
>>   monnier@iro.umontreal.ca
>> From: Dmitry Gutov<dgutov@yandex.ru>
>>
>> On 09/12/2022 22:20, Eli Zaretskii wrote:
>>
>>>> +(defun prog-fill-reindent-defun (&optional argument)
>>>> +  "Refill paragraph or reindent the definition that the point is on.
>>>> +
>>>> +If the point is in a string, or in a comment, or there is a
>>>> +comment on the current line, fill the paragraph that the point is
>>>> +in or is on the same line.
>>> I don't understand the difference between "point is in a string, or in
>>> a comment" and "there is a comment on the current line".
>> The comment may start after point.
> Only after, not before?

Either point is in a comment (possibly multiline), or the comment starts 
after point on the same line.

>>>> +Otherwise, reindent the definition around or below point."
>>> Likewise this one is confusing.
>> The intent was to describe the behavior that follows 'mark-defun'. Which
>> has this in its docstring:
>>
>>     Put mark at end of this defun, point at beginning.
>>     The defun marked is the one that contains point or follows point.
> That one is clear.  Does "below point" means before point (i.e. lower
> buffer position) or after point (i.e. lower on the screen)?

Same as "after point", yes.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2086 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>>  casouri@gmail.com, monnier@iro.umontreal.ca
>> Date: Fri, 09 Dec 2022 22:09:10 +0100
>> 
>> Something like the patch below?  Forgive me, but my texinfo-fu is
>> limited.
>
> You have nothing to worry in that department.
>
>> I'm a little unsure if I xrefed to the correct nodes.  Maybe
>> it is better to just mention 'beginning-of-defun' rather than the
>> variable 'beginning-of-defun-function'?
>
> I meant a reference to where beginning-of-defun-function is mentioned,
> and that is in the ELisp manual.  You can see examples of how to
> reference another manual in many places in doc/emacs/*.texi; seartch
> for "elisp".
>

Ok, thanks!

>> >> +(defun prog-fill-reindent-defun (&optional argument)
>> >> +  "Refill paragraph or reindent the definition that the point is on.
>> >> +
>> >> +If the point is in a string, or in a comment, or there is a
>> >> +comment on the current line, fill the paragraph that the point is
>> >> +in or is on the same line.
>> >
>> > I don't understand the difference between "point is in a string, or in
>> > a comment" and "there is a comment on the current line".  
>> >
>> >> +Otherwise, reindent the definition around or below point."
>> >
>> > Likewise this one is confusing.
>> 
>> I'm not sure how to word this differently, but I agree it's a mouthful.
>
> If you explain to me in plain English what you meant, I can suggest a
> better wording.
>

I made another attempt - I think it's a little clearer.  What I want to
say is something like this (| denotes point):

1. If point is inside a comment, refill the paragraph:
  // foo bar | baz

2. If point is before a comment, refill the paragraph:
 |  // foo bar baz

3. If point is inside code, reindent the defun:
int
main(void)
{
             int i;          // <-- This will be reindented
  /* Some very long comment that will _not_ be refilled in this case.  */
  for(;;)
    {
  |   int x;
    }
}

Anyway, see attached patch:

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 10520 bytes --]

From aeb709da894d19ec4fe5b05be1eb957861eb3dba Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 9 Dec 2022 20:12:51 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c++-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              | 18 ++++++++++++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  4 ++++
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       |  5 ++++
 lisp/progmodes/js.el                 |  5 ++++
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  4 ++++
 lisp/treesit.el                      |  9 +++++++
 10 files changed, 94 insertions(+), 2 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..d85fcd539c 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,9 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+Fill a single paragraph in a defun, or reindent all the lines within
+that defun.
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
@@ -429,6 +432,21 @@ Multi-line Indent
 etc.  To correct the overall indentation as well, type @kbd{@key{TAB}}
 first.
 
+@kindex M-q
+@findex prog-mode-fill-reindent-defun
+@vindex beginning-of-defun-function
+@vindex end-of-defun-function
+@vindex fill-paragraph-function
+  Major modes that derive from @code{prog-mode} can either fill a
+single paragraph in a defun, such as a doc-string, or a comment, or
+(re)indent the surrounding defun if point is not in a comment or a
+string by typing @kbd{M-q}.  The bounds of a defun is decided by the
+variable @code{beginning-of-defun-function} and
+@code{end-of-defun-function}, and the filling mechanism is decided by
+@code{fill-paragraph-function} (@pxref{List Motion,,, elisp, The Emacs
+Lisp Reference Manual} or @ref{Filling,,, elisp, The Emacs Lisp
+Reference Manual} for more information).
+
 @kindex C-u TAB
   If you like the relative indentation within a grouping but not the
 indentation of its first line, move point to that first line and type
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..a5eb7f332d 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a single paragraph in a defun, such as a
+doc-string, or a comment, or (re)indents the surrounding defun if
+point is not in a comment or a string.  It is by default bound to
+'M-q' in 'prog-mode' and all its descendants.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..114ffc0b03 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -603,6 +603,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..b1bf965e4f 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -308,6 +308,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..d0b243a22f 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..1bd8234dc9 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill or reindent the paragraph or defun that contains point.
+
+If the point is in a string or a comment, fill the paragraph that
+contains point or follows point.
+
+Otherwise, reindent the definition that contains point or follows
+point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..59598b4c63 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -323,6 +323,10 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..41feac35c2 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp "\\`comment\\'"
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>  casouri@gmail.com, monnier@iro.umontreal.ca
> Date: Sat, 10 Dec 2022 21:21:53 +0100
> 
> I made another attempt - I think it's a little clearer.  What I want to
> say is something like this (| denotes point):
> 
> 1. If point is inside a comment, refill the paragraph:
>   // foo bar | baz
> 
> 2. If point is before a comment, refill the paragraph:
>  |  // foo bar baz
> 
> 3. If point is inside code, reindent the defun:
> int
> main(void)
> {
>              int i;          // <-- This will be reindented
>   /* Some very long comment that will _not_ be refilled in this case.  */
>   for(;;)
>     {
>   |   int x;
>     }
> }

What if point is after a comment, like this:

   /* foo bar baz */ |

> +@code{fill-paragraph-function} (@pxref{List Motion,,, elisp, The Emacs
> +Lisp Reference Manual} or @ref{Filling,,, elisp, The Emacs Lisp
> +Reference Manual} for more information).

The @pxref should be "see @ref", and please add a comma after the
closing brace (some old versions of makeinfo insist on that).  @pxref
is only appropriate if its closing brace immediately precedes the
closing parenthesis.

Otherwise LGTM, thanks.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1474 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>>  casouri@gmail.com, monnier@iro.umontreal.ca
>> Date: Sat, 10 Dec 2022 21:21:53 +0100
>> 
>> I made another attempt - I think it's a little clearer.  What I want to
>> say is something like this (| denotes point):
>> 
>> 1. If point is inside a comment, refill the paragraph:
>>   // foo bar | baz
>> 
>> 2. If point is before a comment, refill the paragraph:
>>  |  // foo bar baz
>> 
>> 3. If point is inside code, reindent the defun:
>> int
>> main(void)
>> {
>>              int i;          // <-- This will be reindented
>>   /* Some very long comment that will _not_ be refilled in this case.  */
>>   for(;;)
>>     {
>>   |   int x;
>>     }
>> }
>
> What if point is after a comment, like this:
>
>    /* foo bar baz */ |
>

Then the defun will be reindented.  In one line comments the comment
will be filled.


>> +@code{fill-paragraph-function} (@pxref{List Motion,,, elisp, The Emacs
>> +Lisp Reference Manual} or @ref{Filling,,, elisp, The Emacs Lisp
>> +Reference Manual} for more information).
>
> The @pxref should be "see @ref", and please add a comma after the
> closing brace (some old versions of makeinfo insist on that).  @pxref
> is only appropriate if its closing brace immediately precedes the
> closing parenthesis.
>
> Otherwise LGTM, thanks.

Thanks for your patience!

What about this?

Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-prog-fill-reindent-defun-bug-59664.patch --]
[-- Type: text/x-diff, Size: 10572 bytes --]

From f657157b02dd873806d261612bc906f113ea3b89 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Fri, 9 Dec 2022 20:12:51 +0100
Subject: [PATCH] Add prog-fill-reindent-defun (bug#59664)

Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c++-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
---
 doc/emacs/programs.texi              | 19 +++++++++++++++
 etc/NEWS                             |  7 ++++++
 lisp/progmodes/c-ts-mode.el          |  4 ++++
 lisp/progmodes/csharp-mode.el        |  5 ++++
 lisp/progmodes/java-ts-mode.el       |  5 ++++
 lisp/progmodes/js.el                 |  5 ++++
 lisp/progmodes/prog-mode.el          | 35 ++++++++++++++++++++++++++--
 lisp/progmodes/sh-script.el          |  4 ++++
 lisp/progmodes/typescript-ts-mode.el |  4 ++++
 lisp/treesit.el                      |  9 +++++++
 10 files changed, 95 insertions(+), 2 deletions(-)

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index ba8475e86a..3b60732171 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -409,6 +409,9 @@ Multi-line Indent
 @table @kbd
 @item C-M-q
 Reindent all the lines within one parenthetical grouping.
+@item M-q
+Fill a single paragraph in a defun, or reindent all the lines within
+that defun.
 @item C-u @key{TAB}
 Shift an entire parenthetical grouping rigidly sideways so that its
 first line is properly indented.
@@ -429,6 +432,22 @@ Multi-line Indent
 etc.  To correct the overall indentation as well, type @kbd{@key{TAB}}
 first.
 
+@kindex M-q
+@findex prog-fill-reindent-defun
+@vindex beginning-of-defun-function
+@vindex end-of-defun-function
+@vindex fill-paragraph-function
+  Major modes that derive from @code{prog-mode} can either fill a
+single paragraph in a defun, such as a doc-string, or a comment, or
+(re)indent the surrounding defun if point is not in a comment or a
+string by typing @kbd{M-q} or using the command @kbd{M-x
+prog-fill-reindent-defun}.  The bounds of a defun is decided by the
+variable @code{beginning-of-defun-function} and
+@code{end-of-defun-function}, and the filling mechanism is decided by
+@code{fill-paragraph-function} (@ref{List Motion,,, elisp, The Emacs
+Lisp Reference Manual}, or @ref{Filling,,, elisp, The Emacs Lisp
+Reference Manual} for more information).
+
 @kindex C-u TAB
   If you like the relative indentation within a grouping but not the
 indentation of its first line, move point to that first line and type
diff --git a/etc/NEWS b/etc/NEWS
index 61f813568f..a5eb7f332d 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -61,6 +61,13 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 After manually editing 'eshell-aliases-file', you can use
 'M-x eshell-read-aliases-list' to load the edited aliases.
 
+** Prog Mode
++++
+*** New command 'prog-fill-reindent-defun'
+This command either fills a single paragraph in a defun, such as a
+doc-string, or a comment, or (re)indents the surrounding defun if
+point is not in a comment or a string.  It is by default bound to
+'M-q' in 'prog-mode' and all its descendants.
 \f
 * New Modes and Packages in Emacs 30.1
 
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 08b03d5666..114ffc0b03 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -603,6 +603,10 @@ c++-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "raw_string_literal")))
+
   (treesit-parser-create 'cpp)
 
   (setq-local treesit-simple-indent-rules
diff --git a/lisp/progmodes/csharp-mode.el b/lisp/progmodes/csharp-mode.el
index 8ab5fbc91d..3faeb54fc5 100644
--- a/lisp/progmodes/csharp-mode.el
+++ b/lisp/progmodes/csharp-mode.el
@@ -911,6 +911,11 @@ csharp-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "verbatim_string-literal"
+                            "interpolated_verbatim_string-text")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules csharp-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 96e0d5244c..b1bf965e4f 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -308,6 +308,11 @@ java-ts-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("line_comment"
+                            "block_comment"
+                            "text_block")))
+
   ;; Indent.
   (setq-local treesit-simple-indent-rules java-ts-mode--indent-rules)
 
diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 45dfef372c..d0b243a22f 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3857,6 +3857,11 @@ js-ts-mode
                     (group (or (syntax comment-end)
                                (seq (+ "*") "/")))))
     (setq-local comment-multi-line t)
+
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "template_string")))
+
     ;; Electric-indent.
     (setq-local electric-indent-chars
 	        (append "{}():;," electric-indent-chars)) ;FIXME: js2-mode adds "[]*".
diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 58cb48f182..1bd8234dc9 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -30,7 +30,11 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib)
-                   (require 'subr-x))
+                   (require 'subr-x)
+                   (require 'treesit))
+
+(declare-function treesit-parser-list "treesit.c")
+(declare-function treesit-node-type "treesit.c")
 
 (defgroup prog-mode nil
   "Generic programming mode, from which others derive."
@@ -102,7 +106,8 @@ prog-context-menu
 
 (defvar-keymap prog-mode-map
   :doc "Keymap used for programming modes."
-  "C-M-q" #'prog-indent-sexp)
+  "C-M-q" #'prog-indent-sexp
+  "M-q" #'prog-fill-reindent-defun)
 
 (defvar prog-indentation-context nil
   "When non-nil, provides context for indenting embedded code chunks.
@@ -140,6 +145,32 @@ prog-indent-sexp
 	  (end (progn (forward-sexp 1) (point))))
       (indent-region start end nil))))
 
+(defun prog-fill-reindent-defun (&optional argument)
+  "Refill or reindent the paragraph or defun that contains point.
+
+If the point is in a string or a comment, fill the paragraph that
+contains point or follows point.
+
+Otherwise, reindent the definition that contains point or follows
+point."
+  (interactive "P")
+  (save-excursion
+    (let ((treesit-text-node
+           (and (treesit-parser-list)
+                (string-match-p
+                 treesit-text-type-regexp
+                 (treesit-node-type (treesit-node-at (point)))))))
+      (if (or treesit-text-node
+              (nth 8 (syntax-ppss))
+              (re-search-forward comment-start-skip (line-end-position) t))
+          (if (memq fill-paragraph-function '(t nil))
+              (lisp-fill-paragraph argument)
+            (funcall fill-paragraph-function argument))
+        (beginning-of-defun)
+        (let ((start (point)))
+          (end-of-defun)
+          (indent-region start (point) nil))))))
+
 (defun prog-first-column ()
   "Return the indentation column normally used for top-level constructs."
   (or (car prog-indentation-context) 0))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index e170d18afe..1605e40347 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1619,6 +1619,10 @@ bash-ts-mode
                   ( bracket delimiter misc-punctuation operator)))
     (setq-local treesit-font-lock-settings
                 sh-mode--treesit-settings)
+    (setq-local treesit-text-type-regexp
+                (regexp-opt '("comment"
+                              "heredoc_start"
+                              "heredoc_body")))
     (treesit-major-mode-setup)))
 
 (advice-add 'bash-ts-mode :around #'sh--redirect-bash-ts-mode
diff --git a/lisp/progmodes/typescript-ts-mode.el b/lisp/progmodes/typescript-ts-mode.el
index 20916eaf37..59598b4c63 100644
--- a/lisp/progmodes/typescript-ts-mode.el
+++ b/lisp/progmodes/typescript-ts-mode.el
@@ -323,6 +323,10 @@ typescript-ts-base-mode
                   (group (or (syntax comment-end)
                              (seq (+ "*") "/")))))
 
+  (setq-local treesit-text-type-regexp
+              (regexp-opt '("comment"
+                            "template_string")))
+
   ;; Electric
   (setq-local electric-indent-chars
               (append "{}():;," electric-indent-chars))
diff --git a/lisp/treesit.el b/lisp/treesit.el
index dbbf7ec18c..41feac35c2 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1626,6 +1626,15 @@ treesit-end-of-defun
     (when top
       (goto-char (treesit-node-end top)))))
 
+(defvar-local treesit-text-type-regexp "\\`comment\\'"
+  "A regexp that matches the node type of textual nodes.
+
+A textual node is a node that is not normal code, such as
+comments and multiline string literals.  For example,
+\"(line|block)_comment\" in the case of a comment, or
+\"text_block\" in the case of a string.  This is used by
+`prog-fill-reindent-defun' and friends.")
+
 ;;; Activating tree-sitter
 
 (defun treesit-ready-p (language &optional quiet)
-- 
2.34.1

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Eli Zaretskii]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>  casouri@gmail.com, monnier@iro.umontreal.ca
> Date: Sun, 11 Dec 2022 09:39:46 +0100
> 
> What about this?

LGTM, thanks.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 11 December 2022 11:03:35 CET, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Theodor Thornhill <theo@thornhill.no>
>> Cc: dgutov@yandex.ru, larsi@gnus.org, 59662@debbugs.gnu.org,
>>  casouri@gmail.com, monnier@iro.umontreal.ca
>> Date: Sun, 11 Dec 2022 09:39:46 +0100
>> 
>> What about this?
>
>LGTM, thanks.

Nice. Will you or Dmitry install?

Thanks,
Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Dmitry Gutov]

On 11/12/2022 16:13, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> Nice. Will you or Dmitry install?

Pushed to master, thank you.

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/12/2022 16:13, Theodor Thornhill via Bug reports for GNU Emacs, 
> the Swiss army knife of text editors wrote:
>> Nice. Will you or Dmitry install?
>
> Pushed to master, thank you.

Thanks!

Theo

bug#59662: 29.0.50; [PATCH] Add treesit--indent-defun

[Yuan Fu]

Theodor Thornhill <theo@thornhill.no> writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> On 11/12/2022 16:13, Theodor Thornhill via Bug reports for GNU Emacs, 
>> the Swiss army knife of text editors wrote:
>>> Nice. Will you or Dmitry install?
>>
>> Pushed to master, thank you.
>
> Thanks!
>
> Theo

Closing this :-)

Yuan