From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#43389: 28.0.50; Emacs memory leaks using hard disk all time Date: Wed, 25 Nov 2020 20:03:35 +0200 Message-ID: <83blfls494.fsf@gnu.org> References: <86y2j2brg2.fsf@protected.rcdrun.com> <83blfxth7c.fsf@gnu.org> <83y2j0qb2v.fsf@gnu.org> <831rgppg3w.fsf@gnu.org> <83zh3czbvz.fsf@gnu.org> <83blfovzxz.fsf@gnu.org> <87o8jnu5f2.fsf@mail.trevorbentley.com> <83o8jmu49z.fsf@gnu.org> <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27005"; mail-complaints-to="usenet@ciao.gmane.io" Cc: fweimer@redhat.com, 43389@debbugs.gnu.org, bugs@gnu.support, dj@redhat.com, michael_heerdegen@web.de, trevor@trevorbentley.com To: Carlos O'Donell Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Nov 25 19:04:13 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1khz9J-0006ss-5w for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 25 Nov 2020 19:04:13 +0100 Original-Received: from localhost ([::1]:43254 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1khz9I-0003uo-8H for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 25 Nov 2020 13:04:12 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:57214) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1khz9A-0003ue-8d for bug-gnu-emacs@gnu.org; Wed, 25 Nov 2020 13:04:04 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:53583) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1khz98-0006hm-Mm for bug-gnu-emacs@gnu.org; Wed, 25 Nov 2020 13:04:04 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1khz98-0004vQ-Hw for bug-gnu-emacs@gnu.org; Wed, 25 Nov 2020 13:04:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 25 Nov 2020 18:04:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43389 X-GNU-PR-Package: emacs Original-Received: via spool by 43389-submit@debbugs.gnu.org id=B43389.160632742918910 (code B ref 43389); Wed, 25 Nov 2020 18:04:02 +0000 Original-Received: (at 43389) by debbugs.gnu.org; 25 Nov 2020 18:03:49 +0000 Original-Received: from localhost ([127.0.0.1]:36895 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1khz8u-0004uw-FU for submit@debbugs.gnu.org; Wed, 25 Nov 2020 13:03:48 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:38718) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1khz8q-0004uh-1S for 43389@debbugs.gnu.org; Wed, 25 Nov 2020 13:03:47 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:49468) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1khz8k-0006es-9P; Wed, 25 Nov 2020 13:03:38 -0500 Original-Received: from [176.228.60.248] (port=4760 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1khz8h-0002QY-OV; Wed, 25 Nov 2020 13:03:36 -0500 In-Reply-To: <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com> (message from Carlos O'Donell on Wed, 25 Nov 2020 12:45:04 -0500) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:194215 Archived-At: > Cc: bugs@gnu.support, fweimer@redhat.com, 43389@debbugs.gnu.org, > dj@redhat.com, michael_heerdegen@web.de > From: Carlos O'Donell > Date: Wed, 25 Nov 2020 12:45:04 -0500 > > On 11/24/20 11:07 AM, Eli Zaretskii wrote: > > Look at the large chunks in the tail of this. Together, they do > > account for ~2GB. > > > > Carlos, are these chunks in use (i.e. allocated and not freed), or are > > they the free chunks that are available for allocation, but not > > released to the OS? If the former, then it sounds like this session > > does have around 2GB of allocated heap data, so either there's some > > allocated memory we don't account for, or there is indeed a memory > > leak in Emacs. If these are the free chunks, then the way glibc > > manages free'd memory is indeed an issue. > > These chunks are all free and mapped for use by the algorithm to satisfy > a request by the application. So we have more than 1.5GB free memory available for allocation, is that right? But then how to reconcile this with what you say next: > > > => Currently at 4.2GiB in arena 0 (kernel assigned heap). > => The application is using that sbrk'd memory. > > > > > > => This indicates *real* API usage of 4.2GiB. Here you seem to say that these 4.2GB are _used_ by the application? While I thought the large chunks I asked about, which total more than 1.5GB, are a significant part of those 4.2GB? To make sure there are no misunderstandings, I'm talking about this part of the log: [...] If I sum up the "total=" parts of these large numbers, I get 1.6GB. Is this free memory, given back to glibc for future allocations from this arena, and if so, are those 1.6GB part of the 4.2GB total? > This shows the application is USING memory on the main system heap. > > It might not be "leaked" memory since the application might be using it. > > You want visibility into what is USING that memory. > > With glibc-malloc-trace-utils you can try to do that with: > > LD_PRELOAD=libmtrace.so \ > MTRACE_CTL_FILE=/home/user/app.mtr \ > MTRACE_CTL_BACKTRACE=1 \ > ./app > > This will use libgcc's unwinder to get a copy of the malloc caller > address and then we'll have to decode that based on a /proc/self/maps. > > Next steps: > - Get a glibc-malloc-trace-utils trace of the application ratcheting. > - Get a copy of /proc/$PID/maps for the application (shorter version of smaps). > > Then we might be able to correlate where all the kernel heap data went? Thanks for the instructions. Would people please try that and report the results?