all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
@ 2022-11-21  0:53 Yuan Fu
  2022-11-21  6:40 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
  2022-11-21 13:19 ` Eli Zaretskii
  0 siblings, 2 replies; 34+ messages in thread
From: Yuan Fu @ 2022-11-21  0:53 UTC (permalink / raw)
  To: 59426


Emacs crashed on a very large C file when c-ts-mode is on, because
the function building the imenu list tries to walk through the whole
parse tree, and end up recusing ~10k times because of how deep the parse
tree is. These recursive functions should have a built-in limit. Does
Emacs already have some way to determined the max recursion limit on
each system? Or should we come up with some hard-coded numbers?

Yuan





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21  0:53 bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit Yuan Fu
@ 2022-11-21  6:40 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
  2022-11-21  7:38   ` Stefan Kangas
  2022-11-21 13:19 ` Eli Zaretskii
  1 sibling, 1 reply; 34+ messages in thread
From: Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2022-11-21  6:40 UTC (permalink / raw)
  To: Yuan Fu; +Cc: 59426

Yuan Fu <casouri@gmail.com> writes:

> Emacs crashed on a very large C file when c-ts-mode is on, because
> the function building the imenu list tries to walk through the whole
> parse tree, and end up recusing ~10k times because of how deep the parse
> tree is. These recursive functions should have a built-in limit. Does
> Emacs already have some way to determined the max recursion limit on
> each system? Or should we come up with some hard-coded numbers?

We _used_ to have max-specpdl-size, which would act as a limit if the
recursion was surrounded by specbind.

But it was deleted in Emacs 29.  Maybe it should be brought back, for
this and other deeply recursive C functions?





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21  6:40 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
@ 2022-11-21  7:38   ` Stefan Kangas
  2022-11-21 12:00     ` Mattias Engdegård
  0 siblings, 1 reply; 34+ messages in thread
From: Stefan Kangas @ 2022-11-21  7:38 UTC (permalink / raw)
  To: Po Lu, Yuan Fu; +Cc: 59426, Mattias Engdegård

Po Lu via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs@gnu.org> writes:

> We _used_ to have max-specpdl-size, which would act as a limit if the
> recursion was surrounded by specbind.
>
> But it was deleted in Emacs 29.  Maybe it should be brought back, for
> this and other deeply recursive C functions?

Copying in Mattias.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21  7:38   ` Stefan Kangas
@ 2022-11-21 12:00     ` Mattias Engdegård
  2022-11-21 13:55       ` Eli Zaretskii
  2022-11-22  0:27       ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
  0 siblings, 2 replies; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-21 12:00 UTC (permalink / raw)
  To: Stefan Kangas; +Cc: Po Lu, 59426, Yuan Fu

>> We _used_ to have max-specpdl-size, which would act as a limit if the
>> recursion was surrounded by specbind.
>> 
>> But it was deleted in Emacs 29.  Maybe it should be brought back, for
>> this and other deeply recursive C functions?

Highly unlikely. If reasonable use can make C recurse too deeply, then fix that bug instead.
What is the call structure leading to the crash? Is Lisp involved?

(I understand that the tree-sitter branch is very experimental as it doesn't even build on my machine.)






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21  0:53 bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit Yuan Fu
  2022-11-21  6:40 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
@ 2022-11-21 13:19 ` Eli Zaretskii
  2022-11-21 16:52   ` Yuan Fu
  1 sibling, 1 reply; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 13:19 UTC (permalink / raw)
  To: Yuan Fu; +Cc: 59426

> From: Yuan Fu <casouri@gmail.com>
> Date: Sun, 20 Nov 2022 16:53:45 -0800
> 
> 
> Emacs crashed on a very large C file when c-ts-mode is on, because
> the function building the imenu list tries to walk through the whole
> parse tree, and end up recusing ~10k times because of how deep the parse
> tree is. These recursive functions should have a built-in limit. Does
> Emacs already have some way to determined the max recursion limit on
> each system? Or should we come up with some hard-coded numbers?

Is the recursion in our code, or is it in libtree-sitter?

If the former, one solution, albeit a crude one, is to track the recursion
level and error out if it becomes too deep.  Another solution is to handle
the stack in our code, in which case the stack can be allocated on the heap.
Yet another solution is to replace stack-based recursive algorithm with
queue-based iteration, like if you replace depth-first search with
breadth-first search.

I'm sure there are other ideas as well.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 12:00     ` Mattias Engdegård
@ 2022-11-21 13:55       ` Eli Zaretskii
  2022-11-21 14:46         ` Mattias Engdegård
  2022-11-22  0:27       ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
  1 sibling, 1 reply; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 13:55 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: luangruo, 59426, casouri, stefankangas

> Cc: Po Lu <luangruo@yahoo.com>, 59426@debbugs.gnu.org,
>  Yuan Fu <casouri@gmail.com>
> From: Mattias Engdegård <mattiase@acm.org>
> Date: Mon, 21 Nov 2022 13:00:45 +0100
> 
> (I understand that the tree-sitter branch is very experimental as it doesn't even build on my machine.)

It is much more than that: it will be merged to master soon (I actually
expected that to happen yesterday, hint, hint), and will be part of Emacs
29.  So if it doesn't build for you, please hurry and report it as a bug
ASAP, if you cannot fix it yourself.

TIA





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 13:55       ` Eli Zaretskii
@ 2022-11-21 14:46         ` Mattias Engdegård
  2022-11-21 16:43           ` Yuan Fu
  0 siblings, 1 reply; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-21 14:46 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: luangruo, 59426, casouri, stefankangas

21 nov. 2022 kl. 14.55 skrev Eli Zaretskii <eliz@gnu.org>:

> if it doesn't build for you, please hurry and report it as a bug [...]if you cannot fix it yourself.

Good, I've now fixed the C compilation errors when building without tree-sitter.

There are still scores of Elisp byte-compilation warnings but I leave fixing those to the tree-sitter developers.






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 14:46         ` Mattias Engdegård
@ 2022-11-21 16:43           ` Yuan Fu
  2022-11-21 16:54             ` Eli Zaretskii
                               ` (2 more replies)
  0 siblings, 3 replies; 34+ messages in thread
From: Yuan Fu @ 2022-11-21 16:43 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas



> On Nov 21, 2022, at 6:46 AM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
> 21 nov. 2022 kl. 14.55 skrev Eli Zaretskii <eliz@gnu.org>:
> 
>> if it doesn't build for you, please hurry and report it as a bug [...]if you cannot fix it yourself.
> 
> Good, I've now fixed the C compilation errors when building without tree-sitter.

Thanks!

> There are still scores of Elisp byte-compilation warnings but I leave fixing those to the tree-sitter developers.

Could you paste the warnings? Because I don’t see any warnings on my machine.

Yuan






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 13:19 ` Eli Zaretskii
@ 2022-11-21 16:52   ` Yuan Fu
  2022-11-21 17:16     ` Eli Zaretskii
  0 siblings, 1 reply; 34+ messages in thread
From: Yuan Fu @ 2022-11-21 16:52 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 59426



> On Nov 21, 2022, at 5:19 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Sun, 20 Nov 2022 16:53:45 -0800
>> 
>> 
>> Emacs crashed on a very large C file when c-ts-mode is on, because
>> the function building the imenu list tries to walk through the whole
>> parse tree, and end up recusing ~10k times because of how deep the parse
>> tree is. These recursive functions should have a built-in limit. Does
>> Emacs already have some way to determined the max recursion limit on
>> each system? Or should we come up with some hard-coded numbers?
> 
> Is the recursion in our code, or is it in libtree-sitter?

In our code, when we walk the parse tree.

> 
> If the former, one solution, albeit a crude one, is to track the recursion
> level and error out if it becomes too deep.  Another solution is to handle
> the stack in our code, in which case the stack can be allocated on the heap.

That’s my idea, hence my asking for a reasonable way to get a limit. I think a hard limit is totally reasonable, because there is no way for a “normal” parse tree to be 10k levels deep (that means the source program is 10k levels deep, ver unlikely for any program a human would write or a machine would generated). The one I observed is likely due to the parser misunderstanding the source (due to errors in the code). Plus, I don’t think any user would want to walk that deep into the parse tree either. If someone expects to walk that deep into a parse tree, her program is ill-designed.

The tree-walking function already has a limit parameter, we just need to give it a default value.

> Yet another solution is to replace stack-based recursive algorithm with
> queue-based iteration, like if you replace depth-first search with
> breadth-first search.

Because of reasons above, I think a hard limit is good enough.

> 
> I'm sure there are other ideas as well.

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 16:43           ` Yuan Fu
@ 2022-11-21 16:54             ` Eli Zaretskii
  2022-11-21 17:10               ` Eli Zaretskii
  2022-11-21 16:56             ` Mattias Engdegård
  2022-11-22  1:46             ` Stefan Kangas
  2 siblings, 1 reply; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 16:54 UTC (permalink / raw)
  To: Yuan Fu; +Cc: luangruo, mattiase, stefankangas, 59426

> From: Yuan Fu <casouri@gmail.com>
> Date: Mon, 21 Nov 2022 08:43:45 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  Stefan Kangas <stefankangas@gmail.com>,
>  Po Lu <luangruo@yahoo.com>,
>  59426@debbugs.gnu.org
> 
> > There are still scores of Elisp byte-compilation warnings but I leave fixing those to the tree-sitter developers.
> 
> Could you paste the warnings? Because I don’t see any warnings on my machine.

I' see them and am working on fixing them.  (You won't see them unless you
configure --without-tree-sitter.)





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 16:43           ` Yuan Fu
  2022-11-21 16:54             ` Eli Zaretskii
@ 2022-11-21 16:56             ` Mattias Engdegård
  2022-11-21 17:01               ` Yuan Fu
  2022-11-21 17:44               ` Eli Zaretskii
  2022-11-22  1:46             ` Stefan Kangas
  2 siblings, 2 replies; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-21 16:56 UTC (permalink / raw)
  To: Yuan Fu; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas

21 nov. 2022 kl. 17.43 skrev Yuan Fu <casouri@gmail.com>:

>> There are still scores of Elisp byte-compilation warnings but I leave fixing those to the tree-sitter developers.
> 
> Could you paste the warnings? Because I don’t see any warnings on my machine.

Build without tree-sitter and you should see them for several .el files, as well as when byte-compiling treesit-tests.el .
(I could paste the warnings but you need to reproduce them yourself in order to fix them anyway.)






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 16:56             ` Mattias Engdegård
@ 2022-11-21 17:01               ` Yuan Fu
  2022-11-21 17:44               ` Eli Zaretskii
  1 sibling, 0 replies; 34+ messages in thread
From: Yuan Fu @ 2022-11-21 17:01 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas



> On Nov 21, 2022, at 8:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Mon, 21 Nov 2022 08:43:45 -0800
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>> Stefan Kangas <stefankangas@gmail.com>,
>> Po Lu <luangruo@yahoo.com>,
>> 59426@debbugs.gnu.org
>> 
>>> There are still scores of Elisp byte-compilation warnings but I leave fixing those to the tree-sitter developers.
>> 
>> Could you paste the warnings? Because I don’t see any warnings on my machine.
> 
> I' see them and am working on fixing them.  (You won't see them unless you
> configure --without-tree-sitter.)

Thanks!

> On Nov 21, 2022, at 8:56 AM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
> 21 nov. 2022 kl. 17.43 skrev Yuan Fu <casouri@gmail.com>:
> 
>>> There are still scores of Elisp byte-compilation warnings but I leave fixing those to the tree-sitter developers.
>> 
>> Could you paste the warnings? Because I don’t see any warnings on my machine.
> 
> Build without tree-sitter and you should see them for several .el files, as well as when byte-compiling treesit-tests.el .
> (I could paste the warnings but you need to reproduce them yourself in order to fix them anyway.)
> 

Ah, I see. I’ll let Eli do the hard work for me :-)

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 16:54             ` Eli Zaretskii
@ 2022-11-21 17:10               ` Eli Zaretskii
  2022-11-21 17:45                 ` Eli Zaretskii
  0 siblings, 1 reply; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 17:10 UTC (permalink / raw)
  To: casouri; +Cc: luangruo, mattiase, stefankangas, 59426

> Cc: luangruo@yahoo.com, mattiase@acm.org, stefankangas@gmail.com,
>  59426@debbugs.gnu.org
> Date: Mon, 21 Nov 2022 18:54:12 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Yuan Fu <casouri@gmail.com>
> > Date: Mon, 21 Nov 2022 08:43:45 -0800
> > Cc: Eli Zaretskii <eliz@gnu.org>,
> >  Stefan Kangas <stefankangas@gmail.com>,
> >  Po Lu <luangruo@yahoo.com>,
> >  59426@debbugs.gnu.org
> > 
> > > There are still scores of Elisp byte-compilation warnings but I leave fixing those to the tree-sitter developers.
> > 
> > Could you paste the warnings? Because I don’t see any warnings on my machine.
> 
> I' see them and am working on fixing them.  (You won't see them unless you
> configure --without-tree-sitter.)

Done, the warnings should be gone now.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 16:52   ` Yuan Fu
@ 2022-11-21 17:16     ` Eli Zaretskii
  0 siblings, 0 replies; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 17:16 UTC (permalink / raw)
  To: Yuan Fu; +Cc: 59426

> From: Yuan Fu <casouri@gmail.com>
> Date: Mon, 21 Nov 2022 08:52:53 -0800
> Cc: 59426@debbugs.gnu.org
> 
> 
> 
> > On Nov 21, 2022, at 5:19 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri@gmail.com>
> >> Date: Sun, 20 Nov 2022 16:53:45 -0800
> >> 
> >> 
> >> Emacs crashed on a very large C file when c-ts-mode is on, because
> >> the function building the imenu list tries to walk through the whole
> >> parse tree, and end up recusing ~10k times because of how deep the parse
> >> tree is. These recursive functions should have a built-in limit. Does
> >> Emacs already have some way to determined the max recursion limit on
> >> each system? Or should we come up with some hard-coded numbers?
> > 
> > Is the recursion in our code, or is it in libtree-sitter?
> 
> In our code, when we walk the parse tree.
> 
> > 
> > If the former, one solution, albeit a crude one, is to track the recursion
> > level and error out if it becomes too deep.  Another solution is to handle
> > the stack in our code, in which case the stack can be allocated on the heap.
> 
> That’s my idea, hence my asking for a reasonable way to get a limit. I think a hard limit is totally reasonable, because there is no way for a “normal” parse tree to be 10k levels deep (that means the source program is 10k levels deep, ver unlikely for any program a human would write or a machine would generated). The one I observed is likely due to the parser misunderstanding the source (due to errors in the code). Plus, I don’t think any user would want to walk that deep into the parse tree either. If someone expects to walk that deep into a parse tree, her program is ill-designed.

How many bytes does each recursive invocation need on the stack?  With that
number at hand, we can estimate a safe value for the limit.  And don't
forget that GC is also highly recursive and eats up a lot of stack space.
I guess GC can happen during building of the imenu list?





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 16:56             ` Mattias Engdegård
  2022-11-21 17:01               ` Yuan Fu
@ 2022-11-21 17:44               ` Eli Zaretskii
  1 sibling, 0 replies; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 17:44 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: luangruo, 59426, casouri, stefankangas

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Mon, 21 Nov 2022 17:56:01 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Kangas <stefankangas@gmail.com>,
>         Po Lu <luangruo@yahoo.com>, 59426@debbugs.gnu.org
> 
> 21 nov. 2022 kl. 17.43 skrev Yuan Fu <casouri@gmail.com>:
> 
> Build without tree-sitter and you should see them for several .el files, as well as when byte-compiling treesit-tests.el .
> (I could paste the warnings but you need to reproduce them yourself in order to fix them anyway.)

You should have shown the warnings, because reconfiguring is a pain.  And
you should have also said that the problems you see are when building the
branch --without-tree-sitter.  This kind of detail should not be omitted
from a good report.

TIA





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 17:10               ` Eli Zaretskii
@ 2022-11-21 17:45                 ` Eli Zaretskii
  2022-11-21 18:20                   ` Mattias Engdegård
  0 siblings, 1 reply; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 17:45 UTC (permalink / raw)
  To: casouri; +Cc: luangruo, mattiase, stefankangas, 59426

> Cc: luangruo@yahoo.com, mattiase@acm.org, stefankangas@gmail.com,
>  59426@debbugs.gnu.org
> Date: Mon, 21 Nov 2022 19:10:24 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> Done, the warnings should be gone now.

But I didn't touch treesit-tests.el.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 17:45                 ` Eli Zaretskii
@ 2022-11-21 18:20                   ` Mattias Engdegård
  2022-11-21 18:26                     ` Eli Zaretskii
  2022-11-21 19:00                     ` Yuan Fu
  0 siblings, 2 replies; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-21 18:20 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: luangruo, 59426, casouri, stefankangas

> And don't forget that GC is also highly recursive and eats up a lot of stack space.

Not any more.

---

If tree-sitter really needs a deep stack whose size depend on the input file and grammar in an unbounded way, then it shouldn't use the C stack; that would be an unnecessary restriction on the files that could be edited. A dynamically allocated stack could still have a limit, but it would be decoupled from the C stack size.







^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 18:20                   ` Mattias Engdegård
@ 2022-11-21 18:26                     ` Eli Zaretskii
  2022-11-21 18:59                       ` Mattias Engdegård
  2022-11-21 19:00                     ` Yuan Fu
  1 sibling, 1 reply; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-21 18:26 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: luangruo, 59426, casouri, stefankangas

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Mon, 21 Nov 2022 19:20:45 +0100
> Cc: casouri@gmail.com, luangruo@yahoo.com, stefankangas@gmail.com,
>         59426@debbugs.gnu.org
> 
> > And don't forget that GC is also highly recursive and eats up a lot of stack space.
> 
> Not any more.

Damn those changes!  Maybe Emacs is no longer Emacs, either...





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 18:26                     ` Eli Zaretskii
@ 2022-11-21 18:59                       ` Mattias Engdegård
  0 siblings, 0 replies; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-21 18:59 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: luangruo, 59426, casouri, stefankangas

21 nov. 2022 kl. 19.26 skrev Eli Zaretskii <eliz@gnu.org>:

>>> And don't forget that GC is also highly recursive and eats up a lot of stack space.
>> 
>> Not any more.
> 
> Damn those changes!  Maybe Emacs is no longer Emacs, either...

Plus ça change...






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 18:20                   ` Mattias Engdegård
  2022-11-21 18:26                     ` Eli Zaretskii
@ 2022-11-21 19:00                     ` Yuan Fu
  2022-11-22  9:08                       ` Mattias Engdegård
  1 sibling, 1 reply; 34+ messages in thread
From: Yuan Fu @ 2022-11-21 19:00 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: luangruo, 59426, Eli Zaretskii, stefankangas



> On Nov 21, 2022, at 10:20 AM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
>> And don't forget that GC is also highly recursive and eats up a lot of stack space.
> 
> Not any more.
> 
> ---
> 
> If tree-sitter really needs a deep stack whose size depend on the input file and grammar in an unbounded way, then it shouldn't use the C stack; that would be an unnecessary restriction on the files that could be edited. A dynamically allocated stack could still have a limit, but it would be decoupled from the C stack size.

Fortunately tree-sitter doesn’t need a deep stack. I don’t think any human-written or even machine generated source file is ever intended to parse into a tree of more than 1k level. Eg, who would write/generate a function that has thousands level of nested brackets {{{{{{{{{{{{{{{{…. ? (Unless they want to try to break the parser/compiler.) So a sane limit is more than enough, just to guard against weird source files that makes the parser (erroneously) generate very very tall trees.

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 12:00     ` Mattias Engdegård
  2022-11-21 13:55       ` Eli Zaretskii
@ 2022-11-22  0:27       ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
  2022-11-22  8:59         ` Mattias Engdegård
  1 sibling, 1 reply; 34+ messages in thread
From: Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2022-11-22  0:27 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: 59426, Yuan Fu, Stefan Kangas

Mattias Engdegård <mattiase@acm.org> writes:

> Highly unlikely. If reasonable use can make C recurse too deeply, then
> fix that bug instead.

When the use is not reasonable (as C files with thousands of nested
brackets clearly are not), max-specpdl-size can prevent Emacs from
crashing.

> What is the call structure leading to the crash? Is Lisp involved?

Apparently not.  I don't know the details.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 16:43           ` Yuan Fu
  2022-11-21 16:54             ` Eli Zaretskii
  2022-11-21 16:56             ` Mattias Engdegård
@ 2022-11-22  1:46             ` Stefan Kangas
  2 siblings, 0 replies; 34+ messages in thread
From: Stefan Kangas @ 2022-11-22  1:46 UTC (permalink / raw)
  To: Yuan Fu, Mattias Engdegård; +Cc: Po Lu, 59426, Eli Zaretskii

Yuan Fu <casouri@gmail.com> writes:

> Could you paste the warnings? Because I don’t see any warnings on my machine.

In toplevel form:
src/treesit-tests.el:130:28: Warning: Unused lexical variable `pattern'
src/treesit-tests.el:130:36: Warning: Unused lexical variable `doc-node'
src/treesit-tests.el:130:45: Warning: Unused lexical variable `object-node'
src/treesit-tests.el:130:57: Warning: Unused lexical variable `pair-node'
src/treesit-tests.el:181:18: Warning: Unused lexical variable `root-node'
src/treesit-tests.el:181:28: Warning: Unused lexical variable `pattern'
src/treesit-tests.el:181:36: Warning: Unused lexical variable `doc-node'
src/treesit-tests.el:181:45: Warning: Unused lexical variable `object-node'
src/treesit-tests.el:181:57: Warning: Unused lexical variable `pair-node'
src/treesit-tests.el:239:18: Warning: Unused lexical variable `root-node'
src/treesit-tests.el:239:28: Warning: Unused lexical variable `pattern'
src/treesit-tests.el:239:36: Warning: Unused lexical variable `doc-node'
src/treesit-tests.el:239:45: Warning: Unused lexical variable `object-node'
src/treesit-tests.el:239:57: Warning: Unused lexical variable `pair-node'
src/treesit-tests.el:318:18: Warning: Unused lexical variable `root-node'
src/treesit-tests.el:318:28: Warning: Unused lexical variable `pattern'
src/treesit-tests.el:318:36: Warning: Unused lexical variable `doc-node'
src/treesit-tests.el:318:45: Warning: Unused lexical variable `object-node'
src/treesit-tests.el:318:57: Warning: Unused lexical variable `pair-node'
src/treesit-tests.el:374:11: Warning: Unused lexical variable `html'
src/treesit-tests.el:374:23: Warning: Unused lexical variable `html-range'
src/treesit-tests.el:414:28: Warning: Unused lexical variable `doc-node'
src/treesit-tests.el:414:37: Warning: Unused lexical variable `object-node'
src/treesit-tests.el:414:49: Warning: Unused lexical variable `pair-node'
src/treesit-tests.el:426:35: Warning: Unused lexical variable `array-node'
src/treesit-tests.el:476:16: Warning: Unused lexical variable `root-node'





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-22  0:27       ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
@ 2022-11-22  8:59         ` Mattias Engdegård
  0 siblings, 0 replies; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-22  8:59 UTC (permalink / raw)
  To: Po Lu; +Cc: 59426, Yuan Fu, Stefan Kangas

22 nov. 2022 kl. 01.27 skrev Po Lu <luangruo@yahoo.com>:

> When the use is not reasonable (as C files with thousands of nested
> brackets clearly are not), max-specpdl-size can prevent Emacs from
> crashing.

No, that variable is gone.

Using lisp_eval_depth is possible (and used by json for a similar purpose) but an independent limit seems safer, since Lisp evaluation does not necessarily use the C stack.







^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-21 19:00                     ` Yuan Fu
@ 2022-11-22  9:08                       ` Mattias Engdegård
  2022-11-22 23:19                         ` Yuan Fu
  0 siblings, 1 reply; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-22  9:08 UTC (permalink / raw)
  To: Yuan Fu; +Cc: luangruo, 59426, Eli Zaretskii, stefankangas

21 nov. 2022 kl. 20.00 skrev Yuan Fu <casouri@gmail.com>:

> Fortunately tree-sitter doesn’t need a deep stack. I don’t think any human-written or even machine generated source file is ever intended to parse into a tree of more than 1k level. Eg, who would write/generate a function that has thousands level of nested brackets {{{{{{{{{{{{{{{{…. ? (Unless they want to try to break the parser/compiler.) So a sane limit is more than enough, just to guard against weird source files that makes the parser (erroneously) generate very very tall trees.

Thank you, this is good to hear. (Standard minimum limits for languages such as C are quite low; for example see C99 section 5.2.4.1.)

What was the reason for the crash that prompted this bug report? Was it an 'unreasonable' C source file, a grammar mistake (using left recursion where right recursion should have been used, or vice versa), or something else?

I hope that tree-sitter does not require a deep stack to handle C files that are merely very long or has long functions, initialisers etc; this is common for program-generated source code.






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-22  9:08                       ` Mattias Engdegård
@ 2022-11-22 23:19                         ` Yuan Fu
  2022-11-23 10:40                           ` Mattias Engdegård
  0 siblings, 1 reply; 34+ messages in thread
From: Yuan Fu @ 2022-11-22 23:19 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas



> On Nov 22, 2022, at 1:08 AM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
> 21 nov. 2022 kl. 20.00 skrev Yuan Fu <casouri@gmail.com>:
> 
>> Fortunately tree-sitter doesn’t need a deep stack. I don’t think any human-written or even machine generated source file is ever intended to parse into a tree of more than 1k level. Eg, who would write/generate a function that has thousands level of nested brackets {{{{{{{{{{{{{{{{…. ? (Unless they want to try to break the parser/compiler.) So a sane limit is more than enough, just to guard against weird source files that makes the parser (erroneously) generate very very tall trees.
> 
> Thank you, this is good to hear. (Standard minimum limits for languages such as C are quite low; for example see C99 section 5.2.4.1.)
> 
> What was the reason for the crash that prompted this bug report? Was it an 'unreasonable' C source file, a grammar mistake (using left recursion where right recursion should have been used, or vice versa), or something else?

It’s a machine generated file with syntax that tree-sitter-c can’t handle very well. The file is from bug#45248.

> 
> I hope that tree-sitter does not require a deep stack to handle C files that are merely very long or has long functions, initialisers etc; this is common for program-generated source code.

Being merely long is no problem: you’ll get a normal-height but very wide tree. The problem is when the parse tree is very tall. As I mentioned earlier, I don’t think programming language sources would produce ~10k levels nesting under normal circumstances. Having 10k function definitions don’t produce a tall tree, having 10k nested function definitions do. But who would want a program with 10k nested functions?

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-22 23:19                         ` Yuan Fu
@ 2022-11-23 10:40                           ` Mattias Engdegård
  2022-11-23 18:46                             ` Yuan Fu
  0 siblings, 1 reply; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-23 10:40 UTC (permalink / raw)
  To: Yuan Fu; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas

23 nov. 2022 kl. 00.19 skrev Yuan Fu <casouri@gmail.com>:

> It’s a machine generated file with syntax that tree-sitter-c can’t handle very well. The file is from bug#45248.

If you mean https://gitlab.com/wireshark/wireshark/-/raw/master/epan/dissectors/packet-rrc.c, then the parse tree would certainly be wide but not at all tall as far as I can tell. There is a fairly long array initialiser in there, but I've seen worse -- and it shouldn't be treated recursively, should it?







^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-23 10:40                           ` Mattias Engdegård
@ 2022-11-23 18:46                             ` Yuan Fu
  2022-11-23 20:01                               ` Mattias Engdegård
  0 siblings, 1 reply; 34+ messages in thread
From: Yuan Fu @ 2022-11-23 18:46 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas



> On Nov 23, 2022, at 2:40 AM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
> 23 nov. 2022 kl. 00.19 skrev Yuan Fu <casouri@gmail.com>:
> 
>> It’s a machine generated file with syntax that tree-sitter-c can’t handle very well. The file is from bug#45248.
> 
> If you mean https://gitlab.com/wireshark/wireshark/-/raw/master/epan/dissectors/packet-rrc.c, then the parse tree would certainly be wide but not at all tall as far as I can tell. There is a fairly long array initialiser in there, but I've seen worse -- and it shouldn't be treated recursively, should it?

It shouldn’t, but tree-sitter thinks some closing brackets are erroneous and skips them when parsing (it skips erroneous tokens in the hope to parse the rest of the file despite local errors). So a 10k wide tree becomes 10k tall.

We can submit a bug repot to tree-sitter-c (“maybe don’t skip closing brackets even there is error, or somthing”), but that’s another story.

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-23 18:46                             ` Yuan Fu
@ 2022-11-23 20:01                               ` Mattias Engdegård
  2022-11-24  9:17                                 ` Yuan Fu
  0 siblings, 1 reply; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-23 20:01 UTC (permalink / raw)
  To: Yuan Fu; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas

23 nov. 2022 kl. 19.46 skrev Yuan Fu <casouri@gmail.com>:

> It shouldn’t, but tree-sitter thinks some closing brackets are erroneous and skips them when parsing (it skips erroneous tokens in the hope to parse the rest of the file despite local errors). So a 10k wide tree becomes 10k tall.
> 
> We can submit a bug repot to tree-sitter-c (“maybe don’t skip closing brackets even there is error, or somthing”), but that’s another story.

Thanks for the explanation. In this case it seems that it's the #line directive that throws a spanner in the works. You probably already discovered that, but for the record, here is a cut-down example:

static hf_register_info hf[] = {
#line 1 "./asn1/rrc/packet-rrc-hfarr.c"
    { &hf_rrc_DL_DCCH_Message_PDU,
      { "DL-DCCH-Message", "rrc.DL_DCCH_Message_element",
        FT_NONE, BASE_NONE, NULL, 0,
        NULL, HFILL }},
    { &hf_rrc_cellIdentity_c_id,
       {"Cell Identifier", "rrc.cellIdentity.c_id",
       FT_UINT32, BASE_DEC, NULL, 0,
       "The Cell Identifier (C-Id) part of the Cell Identity", HFILL }}
  };

Note how the warning colour of the curly brackets vanishes once the #line line is removed.
Even if this snag is corrected, there will always be cases where preprocessor use causes trouble of this or a similar kind. It seems quite convincing that we should void C recursion in favour of explicit stacks where possible.






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-23 20:01                               ` Mattias Engdegård
@ 2022-11-24  9:17                                 ` Yuan Fu
  2022-11-24 10:24                                   ` Mattias Engdegård
  2022-11-24 10:24                                   ` Eli Zaretskii
  0 siblings, 2 replies; 34+ messages in thread
From: Yuan Fu @ 2022-11-24  9:17 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas



> On Nov 23, 2022, at 12:01 PM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
> 23 nov. 2022 kl. 19.46 skrev Yuan Fu <casouri@gmail.com>:
> 
>> It shouldn’t, but tree-sitter thinks some closing brackets are erroneous and skips them when parsing (it skips erroneous tokens in the hope to parse the rest of the file despite local errors). So a 10k wide tree becomes 10k tall.
>> 
>> We can submit a bug repot to tree-sitter-c (“maybe don’t skip closing brackets even there is error, or somthing”), but that’s another story.
> 
> Thanks for the explanation. In this case it seems that it's the #line directive that throws a spanner in the works. You probably already discovered that, but for the record, here is a cut-down example:
> 
> static hf_register_info hf[] = {
> #line 1 "./asn1/rrc/packet-rrc-hfarr.c"
>    { &hf_rrc_DL_DCCH_Message_PDU,
>      { "DL-DCCH-Message", "rrc.DL_DCCH_Message_element",
>        FT_NONE, BASE_NONE, NULL, 0,
>        NULL, HFILL }},
>    { &hf_rrc_cellIdentity_c_id,
>       {"Cell Identifier", "rrc.cellIdentity.c_id",
>       FT_UINT32, BASE_DEC, NULL, 0,
>       "The Cell Identifier (C-Id) part of the Cell Identity", HFILL }}
>  };
> 
> Note how the warning colour of the curly brackets vanishes once the #line line is removed.
> Even if this snag is corrected, there will always be cases where preprocessor use causes trouble of this or a similar kind. It seems quite convincing that we should void C recursion in favour of explicit stacks where possible.
> 

Does it worth the complexity tho? We only need a stack if we want to support this scenario, in which case tree-sitter has a wrong parse tree. Instead of spending the time and resource to go down that deep tree, it’s better to fail early, and let the user decide to either give up on weird files, or try some other approximation.

It’s too early to tell if being able to go down arbitrarily deep into a deep tree is useful. The only use of traversing the whole tree right now is to generate the imenu indexes, which don’t really need to go down more than 10 levels, since most defun nodes we are interested in are either top-level or near top-level.

So I’d prefer we keep it simple and have a hard limit for now. If we later find that a stack is favorable we can always add it in.

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-24  9:17                                 ` Yuan Fu
@ 2022-11-24 10:24                                   ` Mattias Engdegård
  2022-11-24 19:25                                     ` Yuan Fu
  2022-11-24 10:24                                   ` Eli Zaretskii
  1 sibling, 1 reply; 34+ messages in thread
From: Mattias Engdegård @ 2022-11-24 10:24 UTC (permalink / raw)
  To: Yuan Fu; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas

24 nov. 2022 kl. 10.17 skrev Yuan Fu <casouri@gmail.com>:

> Does it worth the complexity tho? We only need a stack if we want to support this scenario, in which case tree-sitter has a wrong parse tree. Instead of spending the time and resource to go down that deep tree, it’s better to fail early, and let the user decide to either give up on weird files, or try some other approximation.
> 
> It’s too early to tell if being able to go down arbitrarily deep into a deep tree is useful. The only use of traversing the whole tree right now is to generate the imenu indexes, which don’t really need to go down more than 10 levels, since most defun nodes we are interested in are either top-level or near top-level.

You may very well be right about that.

> So I’d prefer we keep it simple and have a hard limit for now. If we later find that a stack is favorable we can always add it in.

That sounds good. Maybe the #line snag in the packet-rrc.c example should be reported upstream?






^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-24  9:17                                 ` Yuan Fu
  2022-11-24 10:24                                   ` Mattias Engdegård
@ 2022-11-24 10:24                                   ` Eli Zaretskii
  1 sibling, 0 replies; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-24 10:24 UTC (permalink / raw)
  To: Yuan Fu; +Cc: luangruo, mattiase, stefankangas, 59426

> From: Yuan Fu <casouri@gmail.com>
> Date: Thu, 24 Nov 2022 01:17:02 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  Po Lu <luangruo@yahoo.com>,
>  Stefan Kangas <stefankangas@gmail.com>,
>  59426@debbugs.gnu.org
> 
> So I’d prefer we keep it simple and have a hard limit for now. If we later find that a stack is favorable we can always add it in.

I tend to agree.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-24 10:24                                   ` Mattias Engdegård
@ 2022-11-24 19:25                                     ` Yuan Fu
  2022-11-24 19:28                                       ` Eli Zaretskii
  0 siblings, 1 reply; 34+ messages in thread
From: Yuan Fu @ 2022-11-24 19:25 UTC (permalink / raw)
  To: Mattias Engdegård; +Cc: Po Lu, 59426, Eli Zaretskii, Stefan Kangas



> On Nov 24, 2022, at 2:24 AM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
> 24 nov. 2022 kl. 10.17 skrev Yuan Fu <casouri@gmail.com>:
> 
>> Does it worth the complexity tho? We only need a stack if we want to support this scenario, in which case tree-sitter has a wrong parse tree. Instead of spending the time and resource to go down that deep tree, it’s better to fail early, and let the user decide to either give up on weird files, or try some other approximation.
>> 
>> It’s too early to tell if being able to go down arbitrarily deep into a deep tree is useful. The only use of traversing the whole tree right now is to generate the imenu indexes, which don’t really need to go down more than 10 levels, since most defun nodes we are interested in are either top-level or near top-level.
> 
> You may very well be right about that.

That brings us back to the original question: what limit should we use? I suggest we use a smaller number since normal tree should never be very tall. Running the following code tells me the height of the tree for xdisp.c is 30:

(defun measure-height (node)
  (let ((children (treesit-node-children node t)))
    (if children
        (1+ (apply #'max (mapcar #'measure-height children)))
      1)))
;; In xdisp.c:
(measure-height (treesit-buffer-root-node))

Maybe a limit in the range of hundreds or thousands? 800?

> 
>> So I’d prefer we keep it simple and have a hard limit for now. If we later find that a stack is favorable we can always add it in.
> 
> That sounds good. Maybe the #line snag in the packet-rrc.c example should be reported upstream?

I stole your sample and opened an issue on their GitHub repo: 

https://github.com/tree-sitter/tree-sitter-c/issues/118

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-24 19:25                                     ` Yuan Fu
@ 2022-11-24 19:28                                       ` Eli Zaretskii
  2022-11-27  2:36                                         ` Yuan Fu
  0 siblings, 1 reply; 34+ messages in thread
From: Eli Zaretskii @ 2022-11-24 19:28 UTC (permalink / raw)
  To: Yuan Fu; +Cc: luangruo, mattiase, stefankangas, 59426

> From: Yuan Fu <casouri@gmail.com>
> Date: Thu, 24 Nov 2022 11:25:58 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  Po Lu <luangruo@yahoo.com>,
>  Stefan Kangas <stefankangas@gmail.com>,
>  59426@debbugs.gnu.org
> 
> That brings us back to the original question: what limit should we use? I suggest we use a smaller number since normal tree should never be very tall. Running the following code tells me the height of the tree for xdisp.c is 30:
> 
> (defun measure-height (node)
>   (let ((children (treesit-node-children node t)))
>     (if children
>         (1+ (apply #'max (mapcar #'measure-height children)))
>       1)))
> ;; In xdisp.c:
> (measure-height (treesit-buffer-root-node))
> 
> Maybe a limit in the range of hundreds or thousands? 800?

I suggest 1000.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
  2022-11-24 19:28                                       ` Eli Zaretskii
@ 2022-11-27  2:36                                         ` Yuan Fu
  0 siblings, 0 replies; 34+ messages in thread
From: Yuan Fu @ 2022-11-27  2:36 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Po Lu, Mattias Engdegård, Stefan Kangas, 59426



> On Nov 24, 2022, at 11:28 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Thu, 24 Nov 2022 11:25:58 -0800
>> Cc: Eli Zaretskii <eliz@gnu.org>,
>> Po Lu <luangruo@yahoo.com>,
>> Stefan Kangas <stefankangas@gmail.com>,
>> 59426@debbugs.gnu.org
>> 
>> That brings us back to the original question: what limit should we use? I suggest we use a smaller number since normal tree should never be very tall. Running the following code tells me the height of the tree for xdisp.c is 30:
>> 
>> (defun measure-height (node)
>>  (let ((children (treesit-node-children node t)))
>>    (if children
>>        (1+ (apply #'max (mapcar #'measure-height children)))
>>      1)))
>> ;; In xdisp.c:
>> (measure-height (treesit-buffer-root-node))
>> 
>> Maybe a limit in the range of hundreds or thousands? 800?
> 
> I suggest 1000.

Ok, I added the default limit in cc086f37e89.

Yuan




^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, other threads:[~2022-11-27  2:36 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-21  0:53 bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit Yuan Fu
2022-11-21  6:40 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
2022-11-21  7:38   ` Stefan Kangas
2022-11-21 12:00     ` Mattias Engdegård
2022-11-21 13:55       ` Eli Zaretskii
2022-11-21 14:46         ` Mattias Engdegård
2022-11-21 16:43           ` Yuan Fu
2022-11-21 16:54             ` Eli Zaretskii
2022-11-21 17:10               ` Eli Zaretskii
2022-11-21 17:45                 ` Eli Zaretskii
2022-11-21 18:20                   ` Mattias Engdegård
2022-11-21 18:26                     ` Eli Zaretskii
2022-11-21 18:59                       ` Mattias Engdegård
2022-11-21 19:00                     ` Yuan Fu
2022-11-22  9:08                       ` Mattias Engdegård
2022-11-22 23:19                         ` Yuan Fu
2022-11-23 10:40                           ` Mattias Engdegård
2022-11-23 18:46                             ` Yuan Fu
2022-11-23 20:01                               ` Mattias Engdegård
2022-11-24  9:17                                 ` Yuan Fu
2022-11-24 10:24                                   ` Mattias Engdegård
2022-11-24 19:25                                     ` Yuan Fu
2022-11-24 19:28                                       ` Eli Zaretskii
2022-11-27  2:36                                         ` Yuan Fu
2022-11-24 10:24                                   ` Eli Zaretskii
2022-11-21 16:56             ` Mattias Engdegård
2022-11-21 17:01               ` Yuan Fu
2022-11-21 17:44               ` Eli Zaretskii
2022-11-22  1:46             ` Stefan Kangas
2022-11-22  0:27       ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors
2022-11-22  8:59         ` Mattias Engdegård
2022-11-21 13:19 ` Eli Zaretskii
2022-11-21 16:52   ` Yuan Fu
2022-11-21 17:16     ` Eli Zaretskii

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.