From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#43389: 28.0.50; Emacs memory leaks using hard disk all time Date: Thu, 26 Nov 2020 15:58:04 +0200 Message-ID: <83pn40qkyb.fsf@gnu.org> References: <83y2j0qb2v.fsf@gnu.org> <831rgppg3w.fsf@gnu.org> <83zh3czbvz.fsf@gnu.org> <83blfovzxz.fsf@gnu.org> <87o8jnu5f2.fsf@mail.trevorbentley.com> <83o8jmu49z.fsf@gnu.org> <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com> <87o8jltglg.fsf@mail.trevorbentley.com> <43b8f55b-d201-76e0-2d19-d97dec8798aa@redhat.com> <87im9ttfeg.fsf@mail.trevorbentley.com> <399d4681-940a-c782-b91e-750e62840cb6@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10228"; mail-complaints-to="usenet@ciao.gmane.io" Cc: fweimer@redhat.com, 43389@debbugs.gnu.org, bugs@gnu.support, dj@redhat.com, michael_heerdegen@web.de, trevor@trevorbentley.com To: Carlos O'Donell Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Nov 26 14:59:23 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kiHnv-0002Va-Fk for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 26 Nov 2020 14:59:23 +0100 Original-Received: from localhost ([::1]:46406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiHnt-000605-Td for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 26 Nov 2020 08:59:21 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53008) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiHna-0005zo-6E for bug-gnu-emacs@gnu.org; Thu, 26 Nov 2020 08:59:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:57017) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kiHnZ-0006VF-VW for bug-gnu-emacs@gnu.org; Thu, 26 Nov 2020 08:59:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kiHnZ-0005VX-VR for bug-gnu-emacs@gnu.org; Thu, 26 Nov 2020 08:59:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 26 Nov 2020 13:59:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43389 X-GNU-PR-Package: emacs Original-Received: via spool by 43389-submit@debbugs.gnu.org id=B43389.160639911621139 (code B ref 43389); Thu, 26 Nov 2020 13:59:01 +0000 Original-Received: (at 43389) by debbugs.gnu.org; 26 Nov 2020 13:58:36 +0000 Original-Received: from localhost ([127.0.0.1]:40330 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiHn9-0005Ut-LK for submit@debbugs.gnu.org; Thu, 26 Nov 2020 08:58:35 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:34686) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiHn4-0005Uc-Ug for 43389@debbugs.gnu.org; Thu, 26 Nov 2020 08:58:34 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:42342) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiHmz-0006K3-6d; Thu, 26 Nov 2020 08:58:25 -0500 Original-Received: from [176.228.60.248] (port=2210 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kiHmx-0004gw-RQ; Thu, 26 Nov 2020 08:58:24 -0500 In-Reply-To: <399d4681-940a-c782-b91e-750e62840cb6@redhat.com> (message from Carlos O'Donell on Wed, 25 Nov 2020 15:51:16 -0500) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:194321 Archived-At: > Cc: Eli Zaretskii , fweimer@redhat.com, 43389@debbugs.gnu.org, > dj@redhat.com, michael_heerdegen@web.de > From: Carlos O'Donell > Date: Wed, 25 Nov 2020 15:51:16 -0500 > > >  The raw massif output: > >  http://trevorbentley.com/massif.out.3364630 > >  The *full* tree output: > >  http://trevorbentley.com/ms_print.3364630.txt > >  The tree output showing only entries above 10% usage: > >  http://trevorbentley.com/ms_print.thresh10.3364630.txt > > This data is pretty clear: > > 1.40GiB - lisp_align_malloc (alloc.c:1195) > 1.40GiB - lmalloc (alloc.c:1359) > 0.65GiB - lrealloc (alloc.c:1374) > 0.24GiB - AcquireAlignedMemory (/usr/lib/libMagickCore-7.Q16HDRI.so.7.0.0) > -------- > 3.60Gib - In use as of the snapshot. > > That's a fairly high fraction of the ~4.2GiB that is eventually in use. > > With lisp_align_malloc, lmalloc, and lrealloc shooting up exponentially at the end of the run look like they are making lists and processing numbers and other objects. > > This is a direct expression of something increasing demand for memory. So, at least in Trevor's case, it sounds like we sometimes request a lot of memory during short periods of time. But what kind of memory is that? lmalloc is called by xmalloc, xrealloc, xzalloc, and xpalloc -- functions Emacs calls to get memory unrelated to Lisp data. But it is also called by lisp_malloc, which is used to allocate memory for some Lisp objects. lisp_align_malloc, OTOH, is used exclusively for allocating Lisp data (conses, strings, etc.). It is somewhat strange that lisp_align_malloc and lmalloc were called to allocate similar amounts of memory: these two functions are orthogonal, AFAICS, used for disparate groups of Lisp object types, and it sounds strange that we somehow allocate very similar amounts of memory for those data types. Another observation is that since GC succeeds to release a large portion of this memory, it would probably mean some significant proportion of the calls are for Lisp data, maybe strings (because GC compacts strings, which can allow Emacs to release more memory to glibc's heap allocation machinery). Apart of that, I think we really need to see the most significant customers of these functions when the memory footprint starts growing fast.