New defun navigation for tree-sitter (Was: Code navigation for sh-mode with Tree-sitter)

[Yuan Fu <casouri@gmail.com>]

Thanks to Alan and João, I have revamped the defun navigation of tree-sitter, now major modes can get accurate defun navigation by setting treesit-defun-type-regexp (as before), and switch between top-level-only mode and nested defun mode by treesit-defun-tactic (new).

The change I pushed doesn’t replace the old functions, my plan is to replace the existing ones if we think this change is ok for the release branch. After experimenting and implementing this, I don’t think the current beginning/end-of-defun can support nested navigation without large changes. So I’d prefer we use new beg/end-of-defun commands in tree-sitter major modes. And we later think about merging the two.

I also added comprehensive tests to the defun navigation[1], which should give us more peace in mind. Although this is a large change, I hope it can stay on the release branch because (1) it fixes current bugs (2) it has a comprehensive test.

Yuan

[1] For tests, see 
treesit-defun-navigation-nested-1
treesit-defun-navigation-nested-2
treesit-defun-navigation-nested-3
treesit-defun-navigation-top-level

It tests all possible defun navigations starting from all possible starting positions (that I can think of), so it should be pretty comprehensive. I currently have test for python, js, and bash. New tests can be easily added.

The test program is like the following, we mark positions with markers like [100]

[100]
[101]class Class1():
[999]    prop = 0
[102]
[103]class Class2():[0]
[104]    [1]def method1():
[999]        [2]return 0[3]
[105]    [4]
[106]    [5]def method2():
[999]        [6]return 0[7]
[107]    [8]
[999]    prop = 0[9]
[108]
[109]class Class3():
[999]    prop = 0[10]
[110]

Then we have a list of positions like this, basically saying “if point is at marker X, and we find the next/previous beginning/end of defun, point should end up at Y”, and we have X for all the possible positions in a nested defun, and Y for all four operations. So the first line says “if we start at marker 0, and find prev-beg-of-defun, we should end up at marker 103”.

;; START PREV-BEG NEXT-END PREV-END NEXT-BEG
  '((0 103 105 102 106) ; Between Beg of parent & 1st sibling.
    (1 103 105 102 106) ; Beg of 1st sibling.
    (2 104 105 102 106) ; Inside 1st sibling.
    (3 104 107 102 109) ; End of 1st sibling.
    (4 104 107 102 109) ; Between 1st sibling & 2nd sibling.
    (5 104 107 102 109) ; Beg of 2nd sibling.
    (6 106 107 105 109) ; Inside 2nd sibling.
    (7 106 108 105 109) ; End of 2nd sibling.
    (8 106 108 105 109) ; Between 2nd sibling & end of parent.
    (9 103 110 102 nil) ; End of parent.

    (100 nil 102 nil 103) ; Before 1st parent.
    (101 nil 102 nil 103) ; Beg of 1st parent.
    (102 101 108 nil 109) ; Between 1st & 2nd parent.
    (103 101 108 nil 109) ; Beg of 2nd parent.
    (110 109 nil 108 nil) ; After 3rd parent.
    )