bug#60653: 30.0.50; tree-sitter: `treesit-[explore/inspect]-mode' can leak memory

bug#60653: 30.0.50; tree-sitter: `treesit-[explore/inspect]-mode' can leak memory

[Mickey Petersen]

If you enable either the `treesit-inspect-mode' or `treesit-explore-mode' and leave them to their devices, you may find that through excessive editing, particularly programmatically, that it can cause memory leaks over time.

I am 90% certain those modes are responsible: if I do not use them, then I do not experience these issues. I have had the oomkiller reap the Emacs process as it simply balloons out of control over time.





In GNU Emacs 30.0.50 (build 6, x86_64-pc-linux-gnu, GTK+ Version
 3.24.20, cairo version 1.16.0) of 2023-01-02 built on mickey-work
Repository revision: c209802f7b3721a1b95113290934a23fee88f678
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12013000
System Description: Ubuntu 20.04.3 LTS

Configured using:
 'configure --with-native-compilation --with-json --with-mailutils
 --without-compress-install --with-imagemagick CC=gcc-10'

bug#60653: 30.0.50; tree-sitter: `treesit-[explore/inspect]-mode' can leak memory

[Yuan Fu]

Mickey Petersen <mickey@masteringemacs.org> writes:

> If you enable either the `treesit-inspect-mode' or
> `treesit-explore-mode' and leave them to their devices, you may find
> that through excessive editing, particularly programmatically, that it
> can cause memory leaks over time.
>
> I am 90% certain those modes are responsible: if I do not use them,
> then I do not experience these issues. I have had the oomkiller reap
> the Emacs process as it simply balloons out of control over time.

Hmmm, maybe it’s just using a lot of memory. Emacs never returns memory
once it allocates them. Are you opening a lot of buffers? A tree-sitter
parse tree uses about 10x memory as the source text.

Yuan

bug#60653: 30.0.50; tree-sitter: `treesit-[explore/inspect]-mode' can leak memory

[Mickey Petersen]

Yuan Fu <casouri@gmail.com> writes:

> Mickey Petersen <mickey@masteringemacs.org> writes:
>
>> If you enable either the `treesit-inspect-mode' or
>> `treesit-explore-mode' and leave them to their devices, you may find
>> that through excessive editing, particularly programmatically, that it
>> can cause memory leaks over time.
>>
>> I am 90% certain those modes are responsible: if I do not use them,
>> then I do not experience these issues. I have had the oomkiller reap
>> the Emacs process as it simply balloons out of control over time.
>
> Hmmm, maybe it’s just using a lot of memory. Emacs never returns memory
> once it allocates them. Are you opening a lot of buffers? A tree-sitter
> parse tree uses about 10x memory as the source text.
>

Nah. This VM's got 64 GiB RAM. So that's not it. And my list of open
buffers does go up or down a bit, but as I'm working on TS stuff, it's
mostly the same handful of smallish files.

Note that I used the third-party tree-sitter implementation for a long
time and never encountered this. And it never seems to happen if I
leave the explorer/inspect mode off. Note that I do do a lot of repeat
editing in the ones that have those modes active.

There's definitely something going on here, and I wish I had more
information to give you, Yuan!

> Yuan

bug#60653: 30.0.50; tree-sitter: `treesit-[explore/inspect]-mode' can leak memory

[Yuan Fu]

Mickey Petersen <mickey@masteringemacs.org> writes:

> Yuan Fu <casouri@gmail.com> writes:
>
>> Mickey Petersen <mickey@masteringemacs.org> writes:
>>
>>> If you enable either the `treesit-inspect-mode' or
>>> `treesit-explore-mode' and leave them to their devices, you may find
>>> that through excessive editing, particularly programmatically, that it
>>> can cause memory leaks over time.
>>>
>>> I am 90% certain those modes are responsible: if I do not use them,
>>> then I do not experience these issues. I have had the oomkiller reap
>>> the Emacs process as it simply balloons out of control over time.
>>
>> Hmmm, maybe it’s just using a lot of memory. Emacs never returns memory
>> once it allocates them. Are you opening a lot of buffers? A tree-sitter
>> parse tree uses about 10x memory as the source text.
>>
>
> Nah. This VM's got 64 GiB RAM. So that's not it. And my list of open
> buffers does go up or down a bit, but as I'm working on TS stuff, it's
> mostly the same handful of smallish files.
>
> Note that I used the third-party tree-sitter implementation for a long
> time and never encountered this. And it never seems to happen if I
> leave the explorer/inspect mode off. Note that I do do a lot of repeat
> editing in the ones that have those modes active.
>
> There's definitely something going on here, and I wish I had more
> information to give you, Yuan!

Thank you. I’ll try to figure it out! At this point my guess is that
there’s some leak in the C code which is triggered a lot by
treesit-explore-mode.

Yuan

bug#60653: 30.0.50; tree-sitter: `treesit-[explore/inspect]-mode' can leak memory

[Yuan Fu]

Yuan Fu <casouri@gmail.com> writes:

> Mickey Petersen <mickey@masteringemacs.org> writes:
>
>> Yuan Fu <casouri@gmail.com> writes:
>>
>>> Mickey Petersen <mickey@masteringemacs.org> writes:
>>>
>>>> If you enable either the `treesit-inspect-mode' or
>>>> `treesit-explore-mode' and leave them to their devices, you may find
>>>> that through excessive editing, particularly programmatically, that it
>>>> can cause memory leaks over time.
>>>>
>>>> I am 90% certain those modes are responsible: if I do not use them,
>>>> then I do not experience these issues. I have had the oomkiller reap
>>>> the Emacs process as it simply balloons out of control over time.
>>>
>>> Hmmm, maybe it’s just using a lot of memory. Emacs never returns memory
>>> once it allocates them. Are you opening a lot of buffers? A tree-sitter
>>> parse tree uses about 10x memory as the source text.
>>>
>>
>> Nah. This VM's got 64 GiB RAM. So that's not it. And my list of open
>> buffers does go up or down a bit, but as I'm working on TS stuff, it's
>> mostly the same handful of smallish files.
>>
>> Note that I used the third-party tree-sitter implementation for a long
>> time and never encountered this. And it never seems to happen if I
>> leave the explorer/inspect mode off. Note that I do do a lot of repeat
>> editing in the ones that have those modes active.
>>
>> There's definitely something going on here, and I wish I had more
>> information to give you, Yuan!
>
> Thank you. I’ll try to figure it out! At this point my guess is that
> there’s some leak in the C code which is triggered a lot by
> treesit-explore-mode.

When you observe the memory leak, do you see warnings mentioning pure
storage, etc? I’m asking because we just fixed a bug that overflows the
pure space which may or may not be the same bug as this one.

Yuan