bug#60623: 30.0.50; Add forward-sentence with tree sitter support

[Theodor Thornhill via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 2037 bytes --]

Daniel Martín <mardani29@yahoo.es> writes:

> Theodor Thornhill via "Bug reports for GNU Emacs, the Swiss army knife
> of text editors" <bug-gnu-emacs@gnu.org> writes:
>
>> Hi all!
>>
>> This patch tweaks the forward-sentence function to be usable with
>> tree-sitter.
>>
>> It follows the same style as the recent change in transpose-sexps, so I
>> hope it isn't too controversial.
>
> Thanks.
>
>>
>> What exact node types do you consider useful for sentence movement?
>>
>
> I haven't thought much about your proposed nodes, I initially thought
> that sentences in a programming language are just "statements".

They aren't really proper propsals.  Mostly some example values to show
that the code works.  The problem with just stating "statements" is that
the names are different across parsers. So in java one would call

```
void foo() {
  var foo = 5;  // <-- This thing
}

```

A "local_variable_declaration" or something like that.  But it would
make sense for M-e to move across that whole line.  So this is language
dependent, I believe.

>
> As a suggestion, treesit-forward-sentence could navigate by textual
> sentences when point is inside comments or strings.
>

Yeah, this is a good idea - added in following patch.

>> +** New defvar-local forward-sentence-function.
>> +The previous implementation of 'forward-sentence' is moved into this
>> +variable, which can be set to customize the sentece movement behavior.
>                                                ^^^^^^^
>                                                sentence
>

Thanks - fixed.

> Also, this feature probably needs an update to the Info documentation to
> mention that Tree-sitter can specialize sentence commands in programming
> modes.

Yes, likely.  I will add this a bit later, when we agree on its behavior
fully :)

@Eli, what doc changes do you see as needed here?

@Juri: I added a change with how I understood what you meant.  Is that
in your line of reasoning?


Theo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-forward-sentence-with-tree-sitter-support-bug-60.patch --]
[-- Type: text/x-patch, Size: 6758 bytes --]

From 62d6f162ba7646783675dc87199ca31691164bf4 Mon Sep 17 00:00:00 2001
From: Theodor Thornhill <theo@thornhill.no>
Date: Sat, 7 Jan 2023 12:44:14 +0100
Subject: [PATCH] Add forward-sentence with tree sitter support (bug#60623)

* etc/NEWS: Mention the new changes.
* lisp/progmodes/c-ts-mode.el (c-ts-base-mode): Set the variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Set the variable.
* lisp/textmodes/paragraphs.el (forward-sentence-default-function):
Move old implementation to its own function.
(forward-sentence-function): New defvar defaulting to old behavior.
(forward-sentence): Use the variable in this function unconditionally.
* lisp/treesit.el (treesit-sentence-type-regexp): New defvar.
(treesit-forward-sentence): New defun.
(treesit-major-mode-setup): Conditionally set
forward-sentence-function.
---
 etc/NEWS                       | 13 +++++++++++++
 lisp/progmodes/c-ts-mode.el    |  8 ++++++++
 lisp/progmodes/java-ts-mode.el |  6 ++++++
 lisp/textmodes/paragraphs.el   | 16 +++++++++++++---
 lisp/treesit.el                | 28 ++++++++++++++++++++++++++++
 5 files changed, 68 insertions(+), 3 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 690e9c3faa9..98f5d71cc90 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -57,6 +57,19 @@ treesit.el now unconditionally sets 'transpose-sexps-function' for all
 Tree-sitter modes.  This functionality utilizes the new
 'transpose-sexps-function'.
 
+** New defvar-local forward-sentence-function.
+The previous implementation of 'forward-sentence' is moved into this
+variable, which can be set to customize the sentence movement
+behavior.
+
+** New defvar-local 'treesit-sentence-type-regexp.
+Similarly to 'treesit-defun-type-regexp', this variable is used to
+navigate sentences in Tree-sitter enabled modes.
+
+** New function 'treesit-forward-sentence'.
+treesit.el now conditionally sets 'forward-sentence-function' for all
+Tree-sitter modes that sets 'treesit-sentence-type-regexp'.
+
 \f
 * Changes in Specialized Modes and Packages in Emacs 30.1
 ---
diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index a22f1f3c44f..dec866f762f 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -736,6 +736,14 @@ c-ts-base-mode
   :syntax-table c-ts-mode--syntax-table
 
   ;; Navigation.
+  (setq-local treesit-sentence-type-regexp
+              (regexp-opt '("statement"
+                            "expression"
+                            "definition"
+                            "specifier"
+                            "declaration"
+                            "comment"
+                            "preproc")))
   (setq-local treesit-defun-type-regexp
               (cons (regexp-opt '("function_definition"
                                   "type_definition"
diff --git a/lisp/progmodes/java-ts-mode.el b/lisp/progmodes/java-ts-mode.el
index 87a4e2b90f8..0e3381e219a 100644
--- a/lisp/progmodes/java-ts-mode.el
+++ b/lisp/progmodes/java-ts-mode.el
@@ -296,6 +296,12 @@ java-ts-mode
               (append "{}():;," electric-indent-chars))
 
   ;; Navigation.
+  (setq-local treesit-sentence-type-regexp
+              (regexp-opt '("statement"
+                            "expression"
+                            "parameters"
+                            "list"
+                            "comment")))
   (setq-local treesit-defun-type-regexp
               (regexp-opt '("method_declaration"
                             "class_declaration"
diff --git a/lisp/textmodes/paragraphs.el b/lisp/textmodes/paragraphs.el
index 73abb155aaa..4580be3450d 100644
--- a/lisp/textmodes/paragraphs.el
+++ b/lisp/textmodes/paragraphs.el
@@ -441,14 +441,12 @@ end-of-paragraph-text
 	  (if (< (point) (point-max))
 	      (end-of-paragraph-text))))))
 
-(defun forward-sentence (&optional arg)
+(defun forward-sentence-default-function (&optional arg)
   "Move forward to next end of sentence.  With argument, repeat.
 When ARG is negative, move backward repeatedly to start of sentence.
 
 The variable `sentence-end' is a regular expression that matches ends of
 sentences.  Also, every paragraph boundary terminates sentences as well."
-  (interactive "^p")
-  (or arg (setq arg 1))
   (let ((opoint (point))
         (sentence-end (sentence-end)))
     (while (< arg 0)
@@ -480,6 +478,18 @@ forward-sentence
     (let ((npoint (constrain-to-field nil opoint t)))
       (not (= npoint opoint)))))
 
+(defvar forward-sentence-function #'forward-sentence-default-function
+  "Function to be used to calculate sentence movements.
+See `forward-sentence-default-function' for behavior description.")
+
+(defun forward-sentence (&optional arg)
+  "Move forward to next end of sentence.  With argument, repeat.
+When ARG is negative, move backward repeatedly to start of sentence.
+Delegates its work to `forward-sentence-function'."
+  (interactive "^p")
+  (or arg (setq arg 1))
+  (funcall forward-sentence-function arg))
+
 (defun count-sentences (start end)
   "Count sentences in current buffer from START to END."
   (let ((sentences 0)
diff --git a/lisp/treesit.el b/lisp/treesit.el
index a7f453a8899..0681e758b37 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1783,6 +1783,32 @@ treesit-end-of-defun
     (when treesit-defun-skipper
       (funcall treesit-defun-skipper))))
 
+(defvar-local treesit-sentence-type-regexp ""
+  "A regexp that matches the node type of sentence nodes.
+
+A sentence node is a node that is bigger than a sexp, and
+delimits larger statements in the source code.  It is, however,
+smaller in scope than defuns.  This is used by
+`treesit-forward-sentence' and friends.")
+
+(defun treesit-forward-sentence (&optional arg)
+  "Tree-sitter `forward-sentence-function' function.
+
+ARG is the same as in `forward-sentence-function'.
+
+If inside comment or other nodes described in
+`treesit-sentence-type-regexp', use
+`forward-sentence-default-function', else move across nodes as
+described by `treesit-sentence-type-regexp'."
+
+  (if (string-match-p
+       treesit-text-type-regexp
+       (treesit-node-type (treesit-node-at (point))))
+      (funcall #'forward-sentence-default-function arg)
+    (funcall
+     (if (> arg 0) #'treesit-end-of-thing #'treesit-beginning-of-thing)
+     treesit-sentence-type-regexp (abs arg))))
+
 (defvar-local treesit-text-type-regexp "\\`comment\\'"
   "A regexp that matches the node type of textual nodes.
 
@@ -2256,6 +2282,8 @@ treesit-major-mode-setup
                 #'treesit-add-log-current-defun))
 
   (setq-local transpose-sexps-function #'treesit-transpose-sexps)
+  (when treesit-sentence-type-regexp
+    (setq-local forward-sentence-function #'treesit-forward-sentence))
 
   ;; Imenu.
   (when treesit-simple-imenu-settings
-- 
2.34.1