From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#43389: 28.0.50; Emacs memory leaks Date: Tue, 17 Nov 2020 17:45:15 +0200 Message-ID: <83r1osq96c.fsf@gnu.org> References: <87r1r5428d.fsf@web.de> <874kmcvlbj.fsf@mail.trevorbentley.com> <83imasb0te.fsf@gnu.org> <871rgzvbme.fsf@mail.trevorbentley.com> <83lff6zm8f.fsf@gnu.org> <838sb1rrar.fsf@gnu.org> <87wnylm3sw.fsf@oldenburg2.str.redhat.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5821"; mail-complaints-to="usenet@ciao.gmane.io" Cc: carlos@redhat.com, 43389@debbugs.gnu.org, dj@redhat.com To: Florian Weimer Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Nov 17 16:46:12 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kf3BJ-0001N0-Sd for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 17 Nov 2020 16:46:09 +0100 Original-Received: from localhost ([::1]:37948 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kf3BI-0003OX-V3 for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 17 Nov 2020 10:46:08 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:55588) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kf3BB-0003OP-VD for bug-gnu-emacs@gnu.org; Tue, 17 Nov 2020 10:46:01 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:49382) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kf3BB-0000qN-L9 for bug-gnu-emacs@gnu.org; Tue, 17 Nov 2020 10:46:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kf3BB-0003aQ-Ib for bug-gnu-emacs@gnu.org; Tue, 17 Nov 2020 10:46:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 17 Nov 2020 15:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43389 X-GNU-PR-Package: emacs Original-Received: via spool by 43389-submit@debbugs.gnu.org id=B43389.160562793012085 (code B ref 43389); Tue, 17 Nov 2020 15:46:01 +0000 Original-Received: (at 43389) by debbugs.gnu.org; 17 Nov 2020 15:45:30 +0000 Original-Received: from localhost ([127.0.0.1]:60928 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kf3Ag-00038M-8U for submit@debbugs.gnu.org; Tue, 17 Nov 2020 10:45:30 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:55356) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kf3Ac-00030G-Tt for 43389@debbugs.gnu.org; Tue, 17 Nov 2020 10:45:28 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:56339) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kf3AX-0000YO-A2; Tue, 17 Nov 2020 10:45:21 -0500 Original-Received: from [176.228.60.248] (port=1300 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kf3AW-0005ys-H0; Tue, 17 Nov 2020 10:45:21 -0500 In-Reply-To: <87wnylm3sw.fsf@oldenburg2.str.redhat.com> (message from Florian Weimer on Mon, 16 Nov 2020 21:42:39 +0100) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:193551 Archived-At: > From: Florian Weimer > Cc: carlos@redhat.com, dj@redhat.com, 43389@debbugs.gnu.org > Date: Mon, 16 Nov 2020 21:42:39 +0100 > > * Eli Zaretskii: > > > For some time (several months, I think) we have reports from Emacs > > users that the memory footprints of their Emacs sessions sometimes > > start growing very quickly, from several hundreds of MBytes to several > > gigabytes in a day or even just few hours, and in some cases causing > > the OOMK to kick in and kill the Emacs process. > > The last time I saw this was a genuine memory leak in the Emacs C code. That's always a possibility. However, 2 aspects of these bug reports seem to hint that there's more here than meets the eye: . the problem happens only to a small number of people, and it is hard to find an area in Emacs that would use memory in some special enough way to happen rarely . the Emacs sessions of the people who reported this would run for many days and even weeks on end with fairly normal memory footprint (around 500MB) that was very stable, and then suddenly begin growing by the minute to 10 or 20 times that > There is an issue with reusing posix_memalign allocations. On my system > (running Emacs 27.1 as supplied by Fedora 32), I only see such > allocations as the backing storage for the glib (sic) slab allocator. (By "backing storage" you mean malloc calls that request large chunks so that malloc obtains the memory from mmap? Or do you mean something else?) Are the problems with posix_memalign also relevant to calls to aligned_alloc? Emacs calls the latter _a_lot_, see lisp_align_malloc. > It gets exercised mostly when creating UI elements, as far as I can > tell. I guess your build uses GTK as the toolkit? > There is actually a glibc patch floating around that fixes the aligned > allocation problem, at some (hopefully limited) performance cost to > aligned allocations. We want to get it reviewed and integrated into > upstream glibc. If there is a working reproducer, we could run it > against a patched glibc. We don't have a reproducer, but several people said that the problem happens to them regularly enough in their normal usage. So I think we can ask them to try a patches glibc and see if the problem goes away. > The other issue we have is that thread counts has exceeded in recent > times more than system memory, and glibc basically scales RSS overhead > with thread count, not memory. A use of libgomp suggests that many > threads might indeed be spawned. If their lifetimes overlap, it would > not be unheard of to end up with some RSS overhead in the order of > peak-usage-per-thread times 8 times the number of hardware threads > supported by the system. Setting MALLOC_ARENA_MAX to a small value > counteracts that, so it's very simple to experiment with it if you have > a working reproducer. "Small value" being something like 2? Emacs doesn't use libgomp, I think that comes from ImageMagick, and most people who reported these problems use Emacs that wasn't built with ImageMagick. The only other source of threads in Emacs I know of is GTK, but AFAIK it starts a small number of them, like 4. In any case, experimenting with MALLOC_ARENA_MAX is easy, so I think we should ask the people who experience this to try that. Any other suggestions or thoughts? Thanks.