bug#43389: 28.0.50; Emacs memory leaks using hard disk all time

[Eli Zaretskii <eliz@gnu.org>]

On November 23, 2020 10:11:22 AM GMT+02:00, Jean Louis <bugs@gnu.support> wrote:
> * Eli Zaretskii <eliz@gnu.org> [2020-11-22 23:17]:
> > > Date: Sun, 22 Nov 2020 22:52:14 +0300
> > > From: Jean Louis <bugs@gnu.support>
> > > Cc: fweimer@redhat.com, 43389@debbugs.gnu.org, dj@redhat.com,
> > >   michael_heerdegen@web.de, trevor@trevorbentley.com,
> carlos@redhat.com
> > > 
> > > I am now following this strategy here:
> > >
> https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Leak-Checking
> > 
> > That uses a different implementation of malloc, so I'm not sure it
> > will help us.
> 
> It will not help if you are able to interpret the PDF reports and you
> do not see anything helpful. If you do interpret those PDF reports
> please tell me as such could be useful to find possible causes or find
> other issues in Emacs.

Granted, I looked at the reports before writing that response.  I don't see there anything related to Emacs code.

> Does this here tells you anything?
> https://gnu.support/files/tmp/2020-11-22/jeprof.26889.0.f.heap.pdf

It says that most of memory was allocated by a subroutine of jemalloc.  As I'm not familiar with how jemalloc works, I see no way for us to draw any significant conclusions from that.

> Does this add module isra.0 inside tells you anything?

AFAIU, it's some internal jemalloc midule.

> I am using dynamic modules like vterm and libpq, can that influence
> memory or create memory leaks?

I have no idea, but I don't think I see any of their functions in these reports.

> What is tst_post_reentrancy_raw, is that something that eats memory?

I don't know.  It's something internal to jemalloc.

> I am still running this session with jemalloc and I wish to see if
> anything will happen that blocks the work similar how it blocks with
> the normal run. This helps slightly in determination. As if run of
> Emacs with jemalloc does not cause problems one time, maybe 2-5 times
> or 10 times, that may be deduce problem to standard malloc and not
> Emacs.

The glibc malloc is the prime suspect anyway.  I don't really believe Emacs had such a glaring memory leak.  So trying different malloc implementations is from my POV waste of time at this stage.

> Then in the next session I will try again the tools as described and
> submit data.
> 
> To help me understand, do you think problem is in Emacs or in glibc
> malloc?

I suspect the problem is in how we use glibc's malloc -- there are some usage patterns that cause glibc to be suboptimal in its memory usage, and I hope we will find ways to fine tune it to our needs.

But that is just a guess, and so I wish you'd use the tools pointed out by Carlos, because they are the most efficient way of collecting evidence that might allow us to make some progress here.

We have the attention of the best experts on the issue; let's use their attention and their time as best as we possibly can.

TIA