bug#68993: treesitter support for forward-sexp-default-function

bug#68993: treesitter support for forward-sexp-default-function

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

'treesit-forward-sentence' supports the node type 'text',
and for matching nodes it uses the fallback to
'forward-sentence-default-function'.

This patch does exactly the same for 'treesit-forward-sexp':
for nodes that match a new node type 'comment',
it uses the fallback to the new function
'forward-sexp-default-function'.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: forward-sexp-default-function.patch --]
[-- Type: text/x-diff, Size: 1880 bytes --]

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 82b2f97b4a5..284c4915f3a 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -2137,7 +2137,10 @@ treesit-forward-sexp
   (interactive "^p")
   (let ((arg (or arg 1))
         (pred (or treesit-sexp-type-regexp 'sexp)))
-    (or (if (> arg 0)
+    (or (when (treesit-node-match-p (treesit-node-at (point)) 'comment t)
+          (funcall #'forward-sexp-default-function arg)
+          t)
+        (if (> arg 0)
             (treesit-end-of-thing pred (abs arg) 'restricted)
           (treesit-beginning-of-thing pred (abs arg) 'restricted))
         ;; If we couldn't move, we should signal an error and report
diff --git a/lisp/emacs-lisp/lisp.el b/lisp/emacs-lisp/lisp.el
index 4b722b4e9a7..d3c3bf55de3 100644
--- a/lisp/emacs-lisp/lisp.el
+++ b/lisp/emacs-lisp/lisp.el
@@ -45,7 +45,12 @@ parens-require-spaces
   :type 'boolean
   :group 'lisp)
 
-(defvar forward-sexp-function nil
+(defun forward-sexp-default-function (&optional arg)
+  "Default function for `forward-sexp-function'."
+  (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
+  (if (< arg 0) (backward-prefix-chars)))
+
+(defvar forward-sexp-function #'forward-sexp-default-function
   ;; FIXME:
   ;; - for some uses, we may want a "sexp-only" version, which only
   ;;   jumps over a well-formed sexp, rather than some dwimish thing
@@ -74,10 +79,7 @@ forward-sexp
                                     "No next sexp"
                                   "No previous sexp"))))
     (or arg (setq arg 1))
-    (if forward-sexp-function
-        (funcall forward-sexp-function arg)
-      (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
-      (if (< arg 0) (backward-prefix-chars)))))
+    (funcall forward-sexp-function arg)))
 
 (defun backward-sexp (&optional arg interactive)
   "Move backward across one balanced expression (sexp).

[-- Attachment #3: Type: text/plain, Size: 483 bytes --]


Maybe the node type 'comment' is not the best name,
but it was intended to allow using the default function
to be able to move with 'M-C-f' in the comments and strings
there tree-sitter has no information.

It makes sense to support the default movement with 'M-C-f'
in the comments and strings of all ts modes.  The second patch
shows how this could be achieved by adding the default
'comment' match to 'treesit-thing-settings' of all modes.
Or maybe this should be customizable?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: treesit-major-mode-setup.patch --]
[-- Type: text/x-diff, Size: 1071 bytes --]

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 82b2f97b4a5..284c4915f3a 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -3054,6 +3057,18 @@ treesit-major-mode-setup
     (setq-local outline-search-function #'treesit-outline-search
                 outline-level #'treesit-outline-level))
 
+  (dolist (parser (treesit-parser-list))
+    (let ((language (treesit-parser-language parser))
+          (comment (regexp-opt '("comment" "string" "string_content"))))
+      (unless (treesit-thing-defined-p 'comment language)
+        (if-let ((l (alist-get language treesit-thing-settings)))
+            (progn
+              (setf (alist-get 'comment l) (list comment))
+              (setf (alist-get language treesit-thing-settings) l))
+          (setq-local treesit-thing-settings
+                      (append `((,language (comment ,comment)))
+                              treesit-thing-settings))))))
+
   ;; Remove existing local parsers.
   (dolist (ov (overlays-in (point-min) (point-max)))
     (when-let ((parser (overlay-get ov 'treesit-parser)))

[-- Attachment #5: Type: text/plain, Size: 389 bytes --]


The third patch demonstrates how it's possible to close bug#67036
that was impossible to fix without more general changes in treesit.el.

The problem is that e.g. Ruby parser to such text:

  hash[:key]

produces such syntax tree:

  (element_reference object: (identifier) [ (simple_symbol) ])

so when point is on [ then 'M-C-f' can't move to ].

This is fixed now by the third patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #6: ruby-ts-mode.patch --]
[-- Type: text/x-diff, Size: 1409 bytes --]

diff --git a/lisp/progmodes/ruby-ts-mode.el b/lisp/progmodes/ruby-ts-mode.el
index 598eaa461ff..4d0ae2e9303 100644
--- a/lisp/progmodes/ruby-ts-mode.el
+++ b/lisp/progmodes/ruby-ts-mode.el
@@ -1170,7 +1170,20 @@ ruby-ts-mode
                                 "global_variable"
                                 )
                                eol)
-                              #'ruby-ts--sexp-p)))))
+                              #'ruby-ts--sexp-p))
+                 (comment ,(lambda (node)
+                             (or (member (treesit-node-type node)
+                                         '("comment" "string_content"))
+                                 (and (member (treesit-node-text node)
+                                              '("[" "]"))
+                                      (equal (treesit-node-type
+                                              (treesit-node-parent node))
+                                             "element_reference"))
+                                 (and (member (treesit-node-text node)
+                                              '("#{" "}"))
+                                      (equal (treesit-node-type
+                                              (treesit-node-parent node))
+                                             "interpolation"))))))))
 
   ;; AFAIK, Ruby can not nest methods
   (setq-local treesit-defun-prefer-top-level nil)

bug#68993: treesitter support for forward-sexp-default-function

[Juri Linkov]

Hi Yuan,

Do you think 'comment' is a suitable name for 'treesit-forward-sexp'?
I'm unsure even if 'text' is a good name for 'treesit-forward-sentence'.
But at least these two should be consistent with each another.

Maybe better to add the prefix 'default-' to both?
This will hint that the default function is used.

Then 'treesit-forward-sentence' will support types
'sentence' and 'default-sentence'.
And 'treesit-forward-sexp' will support types
'sexp' and 'default-sexp'.

> @@ -2137,7 +2137,10 @@ treesit-forward-sexp
>    (interactive "^p")
>    (let ((arg (or arg 1))
>          (pred (or treesit-sexp-type-regexp 'sexp)))
> -    (or (if (> arg 0)
> +    (or (when (treesit-node-match-p (treesit-node-at (point)) 'comment t)
> +          (funcall #'forward-sexp-default-function arg)
> +          t)
> +        (if (> arg 0)
>              (treesit-end-of-thing pred (abs arg) 'restricted)
>            (treesit-beginning-of-thing pred (abs arg) 'restricted))
>          ;; If we couldn't move, we should signal an error and report

bug#68993: treesitter support for forward-sexp-default-function

[Yuan Fu]

> On Feb 10, 2024, at 9:46 AM, Juri Linkov <juri@linkov.net> wrote:
> 
> Hi Yuan,
> 
> Do you think 'comment' is a suitable name for 'treesit-forward-sexp'?
> I'm unsure even if 'text' is a good name for 'treesit-forward-sentence'.
> But at least these two should be consistent with each another.
> 
> Maybe better to add the prefix 'default-' to both?
> This will hint that the default function is used.
> 
> Then 'treesit-forward-sentence' will support types
> 'sentence' and 'default-sentence'.
> And 'treesit-forward-sexp' will support types
> 'sexp' and 'default-sexp'.

First there’s the “text” definition, then treesit-forward-sentence uses it as a heuristic to get better results in comments and strings, rather than the other way around. So for me it doesn’t make much sense to say if “text” is good or bad name for treesit-forward-sentence—it’s not for treesit-forward-sentence to start with.

My suggestion would be for both treesit-forward-sentence and -sexp to use “text” for their heuristic. If someone wants more customized behavior, they can always write a custom forward-sentence/sexp function.

Yuan

bug#68993: treesitter support for forward-sexp-default-function

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 291 bytes --]

> My suggestion would be for both treesit-forward-sentence and -sexp to
> use “text” for their heuristic. If someone wants more customized
> behavior, they can always write a custom forward-sentence/sexp function.

Thanks for the suggestion.  So here is the patch that implements this:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: treesit-forward-sexp-text.patch --]
[-- Type: text/x-diff, Size: 1946 bytes --]

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 1443162f79c..62d797513fc 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -2145,7 +2145,10 @@ treesit-forward-sexp
   (interactive "^p")
   (let ((arg (or arg 1))
         (pred (or treesit-sexp-type-regexp 'sexp)))
-    (or (if (> arg 0)
+    (or (when (treesit-node-match-p (treesit-node-at (point)) 'text t)
+          (funcall #'forward-sexp-default-function arg)
+          t)
+        (if (> arg 0)
             (treesit-end-of-thing pred (abs arg) 'restricted)
           (treesit-beginning-of-thing pred (abs arg) 'restricted))
         ;; If we couldn't move, we should signal an error and report
diff --git a/lisp/emacs-lisp/lisp.el b/lisp/emacs-lisp/lisp.el
index c57b1357f63..4c0f720b1a0 100644
--- a/lisp/emacs-lisp/lisp.el
+++ b/lisp/emacs-lisp/lisp.el
@@ -45,7 +45,12 @@ parens-require-spaces
   :type 'boolean
   :group 'lisp)
 
-(defvar forward-sexp-function nil
+(defun forward-sexp-default-function (&optional arg)
+  "Default function for `forward-sexp-function'."
+  (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
+  (if (< arg 0) (backward-prefix-chars)))
+
+(defvar forward-sexp-function #'forward-sexp-default-function
   ;; FIXME:
   ;; - for some uses, we may want a "sexp-only" version, which only
   ;;   jumps over a well-formed sexp, rather than some dwimish thing
@@ -74,10 +79,9 @@ forward-sexp
                                     "No next sexp"
                                   "No previous sexp"))))
     (or arg (setq arg 1))
-    (if forward-sexp-function
-        (funcall forward-sexp-function arg)
-      (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
-      (if (< arg 0) (backward-prefix-chars)))))
+    (funcall (or forward-sexp-function
+                 #'forward-sexp-default-function)
+             arg)))
 
 (defun backward-sexp (&optional arg interactive)
   "Move backward across one balanced expression (sexp).

bug#68993: treesitter support for forward-sexp-default-function

[Yuan Fu]

> On Feb 8, 2024, at 9:38 AM, Juri Linkov <juri@linkov.net> wrote:
> 
> 'treesit-forward-sentence' supports the node type 'text',
> and for matching nodes it uses the fallback to
> 'forward-sentence-default-function'.
> 
> This patch does exactly the same for 'treesit-forward-sexp':
> for nodes that match a new node type 'comment',
> it uses the fallback to the new function
> 'forward-sexp-default-function'.
> 
> diff --git a/lisp/treesit.el b/lisp/treesit.el
> index 82b2f97b4a5..284c4915f3a 100644
> --- a/lisp/treesit.el
> +++ b/lisp/treesit.el
> @@ -2137,7 +2137,10 @@ treesit-forward-sexp
>   (interactive "^p")
>   (let ((arg (or arg 1))
>         (pred (or treesit-sexp-type-regexp 'sexp)))
> -    (or (if (> arg 0)
> +    (or (when (treesit-node-match-p (treesit-node-at (point)) 'comment t)
> +          (funcall #'forward-sexp-default-function arg)
> +          t)
> +        (if (> arg 0)
>             (treesit-end-of-thing pred (abs arg) 'restricted)
>           (treesit-beginning-of-thing pred (abs arg) 'restricted))
>         ;; If we couldn't move, we should signal an error and report
> diff --git a/lisp/emacs-lisp/lisp.el b/lisp/emacs-lisp/lisp.el
> index 4b722b4e9a7..d3c3bf55de3 100644
> --- a/lisp/emacs-lisp/lisp.el
> +++ b/lisp/emacs-lisp/lisp.el
> @@ -45,7 +45,12 @@ parens-require-spaces
>   :type 'boolean
>   :group 'lisp)
> 
> -(defvar forward-sexp-function nil
> +(defun forward-sexp-default-function (&optional arg)
> +  "Default function for `forward-sexp-function'."
> +  (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
> +  (if (< arg 0) (backward-prefix-chars)))
> +
> +(defvar forward-sexp-function #'forward-sexp-default-function
>   ;; FIXME:
>   ;; - for some uses, we may want a "sexp-only" version, which only
>   ;;   jumps over a well-formed sexp, rather than some dwimish thing
> @@ -74,10 +79,7 @@ forward-sexp
>                                     "No next sexp"
>                                   "No previous sexp"))))
>     (or arg (setq arg 1))
> -    (if forward-sexp-function
> -        (funcall forward-sexp-function arg)
> -      (goto-char (or (scan-sexps (point) arg) (buffer-end arg)))
> -      (if (< arg 0) (backward-prefix-chars)))))
> +    (funcall forward-sexp-function arg)))
> 
> (defun backward-sexp (&optional arg interactive)
>   "Move backward across one balanced expression (sexp).
> 
> Maybe the node type 'comment' is not the best name,
> but it was intended to allow using the default function
> to be able to move with 'M-C-f' in the comments and strings
> there tree-sitter has no information.
> 
> It makes sense to support the default movement with 'M-C-f'
> in the comments and strings of all ts modes.  The second patch
> shows how this could be achieved by adding the default
> 'comment' match to 'treesit-thing-settings' of all modes.
> Or maybe this should be customizable?

I think treesit-thing-settings is something we want to left for major mode’s to set. They’ll need to define other “things” in treesit-thing-settings anyway. Sure, it’s nice if we can set a few definitions automatically, but I don’t think the gain is worth that much; OTOH, it’s nice to have clear boundaries, and minimizes the possibility of confusion.

> 
> diff --git a/lisp/treesit.el b/lisp/treesit.el
> index 82b2f97b4a5..284c4915f3a 100644
> --- a/lisp/treesit.el
> +++ b/lisp/treesit.el
> @@ -3054,6 +3057,18 @@ treesit-major-mode-setup
>     (setq-local outline-search-function #'treesit-outline-search
>                 outline-level #'treesit-outline-level))
> 
> +  (dolist (parser (treesit-parser-list))
> +    (let ((language (treesit-parser-language parser))
> +          (comment (regexp-opt '("comment" "string" "string_content"))))
> +      (unless (treesit-thing-defined-p 'comment language)
> +        (if-let ((l (alist-get language treesit-thing-settings)))
> +            (progn
> +              (setf (alist-get 'comment l) (list comment))
> +              (setf (alist-get language treesit-thing-settings) l))
> +          (setq-local treesit-thing-settings
> +                      (append `((,language (comment ,comment)))
> +                              treesit-thing-settings))))))
> +
>   ;; Remove existing local parsers.
>   (dolist (ov (overlays-in (point-min) (point-max)))
>     (when-let ((parser (overlay-get ov 'treesit-parser)))
> 
> The third patch demonstrates how it's possible to close bug#67036
> that was impossible to fix without more general changes in treesit.el.
> 
> The problem is that e.g. Ruby parser to such text:
> 
>  hash[:key]
> 
> produces such syntax tree:
> 
>  (element_reference object: (identifier) [ (simple_symbol) ])
> 
> so when point is on [ then 'M-C-f' can't move to ].
> 
> This is fixed now by the third patch:
> 
> diff --git a/lisp/progmodes/ruby-ts-mode.el b/lisp/progmodes/ruby-ts-mode.el
> index 598eaa461ff..4d0ae2e9303 100644
> --- a/lisp/progmodes/ruby-ts-mode.el
> +++ b/lisp/progmodes/ruby-ts-mode.el
> @@ -1170,7 +1170,20 @@ ruby-ts-mode
>                                 "global_variable"
>                                 )
>                                eol)
> -                              #'ruby-ts--sexp-p)))))
> +                              #'ruby-ts--sexp-p))
> +                 (comment ,(lambda (node)
> +                             (or (member (treesit-node-type node)
> +                                         '("comment" "string_content"))
> +                                 (and (member (treesit-node-text node)
> +                                              '("[" "]"))
> +                                      (equal (treesit-node-type
> +                                              (treesit-node-parent node))
> +                                             "element_reference"))
> +                                 (and (member (treesit-node-text node)
> +                                              '("#{" "}"))
> +                                      (equal (treesit-node-type
> +                                              (treesit-node-parent node))
> +                                             "interpolation"))))))))
> 
>   ;; AFAIK, Ruby can not nest methods
>   (setq-local treesit-defun-prefer-top-level nil)

IIUC, this doesn’t look like a good idea: you don’t want to mark something that’s not comment as comment. In the future, other packages will start using these thing definitions, and I’m sure you don’t want them consider regular code as comments.

For the specific problem you described, maybe the change made in #68899 can help?

Yuan

bug#68993: treesitter support for forward-sexp-default-function

[Juri Linkov]

>> The problem is that e.g. Ruby parser to such text:
>>
>>  hash[:key]
>>
>> produces such syntax tree:
>>
>>  (element_reference object: (identifier) [ (simple_symbol) ])
>>
>> so when point is on [ then 'C-M-f' can't move to ].
>>
>> This is fixed now by the third patch:
>> @@ -1170,7 +1170,20 @@ ruby-ts-mode
>> +                 (comment ,(lambda (node)
>> +                             (or (member (treesit-node-type node)
>> +                                         '("comment" "string_content"))
>> +                                 (and (member (treesit-node-text node)
>> +                                              '("[" "]"))
>> +                                      (equal (treesit-node-type
>> +                                              (treesit-node-parent node))
>> +                                             "element_reference"))
>> +                                 (and (member (treesit-node-text node)
>> +                                              '("#{" "}"))
>> +                                      (equal (treesit-node-type
>> +                                              (treesit-node-parent node))
>> +                                             "interpolation"))))))))
>
> IIUC, this doesn’t look like a good idea: you don’t want to mark
> something that’s not comment as comment. In the future, other packages
> will start using these thing definitions, and I’m sure you don’t want
> them consider regular code as comments.
>
> For the specific problem you described, maybe the change made in #68899 can help?

bug#68899 can't help because it only fixes 'C-M-f' to move point to the
end of the current symbol.

Whereas for the problem above we need a predicate that will instruct
'treesit-forward-sexp' to fall back to 'forward-sexp-default-function'.

Clearly neither the name 'text' nor 'comment' would be suitable
for such usage.  Maybe it's possible to find a different name?

bug#68993: treesitter support for forward-sexp-default-function

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 61 bytes --]

Looks like 568c174135 broke lisp-tests and thingatpt-tests:


[-- Attachment #2: lisp-tests.log --]
[-- Type: application/octet-stream, Size: 5582 bytes --]

  GEN      lisp/emacs-lisp/lisp-tests.log
Running 37 tests (2024-04-16 12:12:01+0200, selector `(not (or (tag :unstable) (tag :nativecomp)))')
   passed   1/37  backward-up-list-basic (0.000521 sec)
   passed   2/37  core-elisp-tests-1-defvar-in-let (0.000078 sec)
Mark set
   passed   3/37  core-elisp-tests-2-window-configurations (0.000078 sec)
   passed   4/37  core-elisp-tests-3-backquote (0.000045 sec)
   passed   5/37  end-of-defun-twice (0.000316 sec)
   passed   6/37  lisp-backward-sexp-1-empty-parens (0.000057 sec)
   passed   7/37  lisp-backward-sexp-1-eobp (0.000056 sec)
   passed   8/37  lisp-backward-sexp-1-error-mismatch (0.000046 sec)
   passed   9/37  lisp-backward-sexp-2-bobp (0.000045 sec)
   passed  10/37  lisp-backward-sexp-2-bobp-and-subsequent (0.000046 sec)
   passed  11/37  lisp-delete-pair-parens (1.005549 sec)
   passed  12/37  lisp-delete-pair-quotation-marks (1.005471 sec)
   passed  13/37  lisp-delete-pair-quotes-in-text-mode (0.000220 sec)
   passed  14/37  lisp-delete-pair-quotes-text-mode-syntax-table (1.005522 sec)
   passed  15/37  lisp-fill-paragraph-colon (0.001064 sec)
   passed  16/37  lisp-forward-sexp-1-empty-parens (0.000099 sec)
   passed  17/37  lisp-forward-sexp-1-eobp (0.000085 sec)
   passed  18/37  lisp-forward-sexp-1-error-mismatch (0.000086 sec)
   passed  19/37  lisp-forward-sexp-2-eobp (0.000085 sec)
   passed  20/37  lisp-forward-sexp-2-eobp-and-subsequent (0.000097 sec)
   passed  21/37  lisp-forward-sexp-elisp-inside-symbol (0.000240 sec)
   passed  22/37  lisp-forward-sexp-elisp-quoted-symbol (0.000221 sec)
   passed  23/37  lisp-forward-sexp-emacs-lisp-quote-char (0.000209 sec)
   passed  24/37  lisp-forward-sexp-emacs-lisp-semi-char-error (0.000213 sec)
   passed  25/37  lisp-forward-sexp-python-triple-quoted-string (0.000601 sec)
   passed  26/37  lisp-forward-sexp-python-triple-quotes-string (0.000365 sec)
   passed  27/37  mark-defun-arg-region-active (0.000385 sec)
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
   passed  28/37  mark-defun-bob (0.000846 sec)
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
   passed  29/37  mark-defun-neg-arg-region-inactive (0.057865 sec)
   passed  30/37  mark-defun-no-arg-region-active (0.000312 sec)
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
   passed  31/37  mark-defun-no-arg-region-inactive (0.000903 sec)
Mark set
Mark set
Mark set
Mark set
Mark set
Mark set
   passed  32/37  mark-defun-pos-arg-region-inactive (0.000503 sec)
   passed  33/37  up-list-basic (0.000085 sec)
Test up-list-cross-string backtrace:
  signal(ert-test-failed (((should (eql (point) (posof instruction))) 
  ert-fail(((should (eql (point) (posof instruction))) :form (eql 24 2
  lisp-run-up-list-test(#f(compiled-function () #<bytecode 0x51b5696be
  #f(compiled-function () #<bytecode 0x515baeca3b92b8a>)()
  #f(compiled-function () #<bytecode 0x18e9853ee4bdb3eb>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x18e9853ee4bdb3eb
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name up-list-cross-string :documentation n
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
  ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
  ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
  eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
  command-line-1(("-L" ":../../emacs/test" "-l" "ert" "--eval" "(setq 
  command-line()
  normal-top-level()
Test up-list-cross-string condition:
    (ert-test-failed
     ((should (eql (point) (posof instruction))) :form (eql 24 21) :value
      nil))
   FAILED  34/37  up-list-cross-string (0.000112 sec) at ../../emacs/test/lisp/emacs-lisp/lisp-tests.el:291
   passed  35/37  up-list-no-cross-string (0.000155 sec)
Test up-list-out-of-string backtrace:
  signal(ert-test-failed (((should (eql (point) (posof instruction))) 
  ert-fail(((should (eql (point) (posof instruction))) :form (eql 23 1
  lisp-run-up-list-test(#f(compiled-function () #<bytecode 0x51b5696be
  #f(compiled-function () #<bytecode -0x1189b0e3987319ce>)()
  #f(compiled-function () #<bytecode 0x18e9853ee4bdb3eb>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x18e9853ee4bdb3eb
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name up-list-out-of-string :documentation 
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
  ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
  ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
  eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
  command-line-1(("-L" ":../../emacs/test" "-l" "ert" "--eval" "(setq 
  command-line()
  normal-top-level()
Test up-list-out-of-string condition:
    (ert-test-failed
     ((should (eql (point) (posof instruction))) :form (eql 23 18) :value
      nil))
   FAILED  36/37  up-list-out-of-string (0.000155 sec) at ../../emacs/test/lisp/emacs-lisp/lisp-tests.el:285
   passed  37/37  up-list-with-forward-sexp-function (0.000164 sec)

Ran 37 tests, 35 results as expected, 2 unexpected (2024-04-16 12:12:04+0200, 3.502098 sec)

2 unexpected results:
   FAILED  up-list-cross-string
   FAILED  up-list-out-of-string

[-- Attachment #3: thingatpt-tests.log --]
[-- Type: application/octet-stream, Size: 2321 bytes --]

  GEN      lisp/thingatpt-tests.log
Running 13 tests (2024-04-16 12:13:44+0200, selector `(not (or (tag :unstable) (tag :nativecomp)))')
   passed   1/13  test-numbers-decimal (0.000164 sec)
   passed   2/13  test-numbers-hex-c (0.000162 sec)
   passed   3/13  test-numbers-hex-lisp (0.000140 sec)
   passed   4/13  test-numbers-none (0.000057 sec)
   passed   5/13  test-symbol-thing-1 (0.000108 sec)
   passed   6/13  test-symbol-thing-2 (0.000067 sec)
   passed   7/13  test-symbol-thing-3 (0.000055 sec)
   passed   8/13  test-symbol-thing-4 (0.000053 sec)
Test thing-at-point-bounds-of-list-at-point backtrace:
  signal(ert-test-failed (((should (equal (list-at-point) (cl-second t
  ert-fail(((should (equal (list-at-point) (cl-second test))) :form (e
  #f(compiled-function () #<bytecode 0xe6420b0d408fe48>)()
  #f(compiled-function () #<bytecode 0x3b8a42f521d079f>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x3b8a42f521d079f>
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name thing-at-point-bounds-of-list-at-poin
  ert-run-or-rerun-test(#s(ert--stats :selector (not ...) :tests [... 
  ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
  ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
  ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
  eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
  command-line-1(("-L" ":../../emacs/test" "-l" "ert" "--eval" "(setq 
  command-line()
  normal-top-level()
Test thing-at-point-bounds-of-list-at-point condition:
    (ert-test-failed
     ((should (equal (list-at-point) (cl-second test))) :form
      (equal nil (a "b" c)) :value nil :explanation
      (different-types nil (a "b" c))))
   FAILED   9/13  thing-at-point-bounds-of-list-at-point (0.000358 sec) at ../../emacs/test/lisp/thingatpt-tests.el:128
   passed  10/13  thing-at-point-looking-at (0.000125 sec)
   passed  11/13  thing-at-point-looking-at-overlapping-matches (0.000123 sec)
   passed  12/13  thing-at-point-tests (0.034991 sec)
   passed  13/13  thing-at-point-url-in-comment (0.081406 sec)

Ran 13 tests, 12 results as expected, 1 unexpected (2024-04-16 12:13:44+0200, 0.341416 sec)

1 unexpected results:
   FAILED  thing-at-point-bounds-of-list-at-point

bug#68993: treesitter support for forward-sexp-default-function

[Juri Linkov]

> Looks like 568c174135 broke lisp-tests and thingatpt-tests:

Thanks, should be fixed now.

bug#68993: treesitter support for forward-sexp-default-function

[Juri Linkov]

close 68993 30.0.50
thanks

>>> My suggestion would be for both treesit-forward-sentence and -sexp to
>>> use “text” for their heuristic. If someone wants more customized
>>> behavior, they can always write a custom forward-sentence/sexp function.
>>
>> Thanks for the suggestion.  So here is the patch that implements this:
>
> LGTM

Thanks, so now pushed and closed.