From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Florian Weimer Newsgroups: gmane.emacs.bugs Subject: bug#43389: 28.0.50; Emacs memory leaks Date: Tue, 17 Nov 2020 17:33:13 +0100 Message-ID: <87o8jwj646.fsf@oldenburg2.str.redhat.com> References: <87r1r5428d.fsf@web.de> <874kmcvlbj.fsf@mail.trevorbentley.com> <83imasb0te.fsf@gnu.org> <871rgzvbme.fsf@mail.trevorbentley.com> <83lff6zm8f.fsf@gnu.org> <838sb1rrar.fsf@gnu.org> <87wnylm3sw.fsf@oldenburg2.str.redhat.com> <83r1osq96c.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38605"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Cc: carlos@redhat.com, 43389@debbugs.gnu.org, dj@redhat.com To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Nov 17 17:34:53 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kf3wT-0009sN-1I for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 17 Nov 2020 17:34:53 +0100 Original-Received: from localhost ([::1]:43746 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kf3wS-0006ON-2H for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 17 Nov 2020 11:34:52 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:38620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kf3ve-0005Bl-M8 for bug-gnu-emacs@gnu.org; Tue, 17 Nov 2020 11:34:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:49444) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kf3ve-0000KK-BA for bug-gnu-emacs@gnu.org; Tue, 17 Nov 2020 11:34:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kf3ve-0005xW-8V for bug-gnu-emacs@gnu.org; Tue, 17 Nov 2020 11:34:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Florian Weimer Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 17 Nov 2020 16:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43389 X-GNU-PR-Package: emacs Original-Received: via spool by 43389-submit@debbugs.gnu.org id=B43389.160563080522858 (code B ref 43389); Tue, 17 Nov 2020 16:34:02 +0000 Original-Received: (at 43389) by debbugs.gnu.org; 17 Nov 2020 16:33:25 +0000 Original-Received: from localhost ([127.0.0.1]:60989 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kf3v3-0005wa-Hv for submit@debbugs.gnu.org; Tue, 17 Nov 2020 11:33:25 -0500 Original-Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39940) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kf3v1-0005wS-BV for 43389@debbugs.gnu.org; Tue, 17 Nov 2020 11:33:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605630803; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=USbkBOrWtA4dZ/Rfoh/2OrW8N4SqeW36zX6kNsB7MJE=; b=AHdvNpqaosHHWePPmID1XB3mOKBFMYm2qMC0F7PgjOT6FsVd3h8CTGR2TpL62VlYY/IWvk za59RGpRwFjWAZnBCYf4uCaHzSyvLa9+WXZ7isivhKazH+IvGlRAodgWfKqvQiJO23eRK7 IC4104xWyB+m+ZD2oZ0loTtNsfdJLjg= Original-Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-231-KlgeTIeLPc28UF0Nr7-swg-1; Tue, 17 Nov 2020 11:33:21 -0500 X-MC-Unique: KlgeTIeLPc28UF0Nr7-swg-1 Original-Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F23388CD88D; Tue, 17 Nov 2020 16:33:19 +0000 (UTC) Original-Received: from oldenburg2.str.redhat.com (ovpn-113-222.ams2.redhat.com [10.36.113.222]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 824285D9CC; Tue, 17 Nov 2020 16:33:15 +0000 (UTC) In-Reply-To: <83r1osq96c.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 17 Nov 2020 17:45:15 +0200") X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=fweimer@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:193555 Archived-At: * Eli Zaretskii: >> There is an issue with reusing posix_memalign allocations. On my system >> (running Emacs 27.1 as supplied by Fedora 32), I only see such >> allocations as the backing storage for the glib (sic) slab allocator. > > (By "backing storage" you mean malloc calls that request large chunks > so that malloc obtains the memory from mmap? Or do you mean something > else?) Larger chunks that are split up by the glib allocator. Whether they are allocated by mmap is unclear. > Are the problems with posix_memalign also relevant to calls to > aligned_alloc? Emacs calls the latter _a_lot_, see lisp_align_malloc. Ahh. I don't see many such calls, even during heavy Gnus usage. But opening really large groups triggers such calls. aligned_alloc is equally problematic. I don't know if the Emacs allocation pattern triggers the pathological behavior. I seem to suffer from the problem as well. glibc malloc currently maintains more than 200 MiB of unused memory: Total RSS is 1 GiB, but even 1 GiB minus 200 MiB would be excessive. It's possible to generate such statistics using GDB, by calling the malloc_info function. My Emacs process does not look like it suffered from the aligned_alloc issue. It would leave behind many smaller, unused allocations, not such large ones. >> It gets exercised mostly when creating UI elements, as far as I can >> tell. > > I guess your build uses GTK as the toolkit? I think so: GNU Emacs 27.1 (build 1, x86_64-redhat-linux-gnu, GTK+ Version 3.24.21, cairo version 1.16.0) of 2020-08-20 >> The other issue we have is that thread counts has exceeded in recent >> times more than system memory, and glibc basically scales RSS overhead >> with thread count, not memory. A use of libgomp suggests that many >> threads might indeed be spawned. If their lifetimes overlap, it would >> not be unheard of to end up with some RSS overhead in the order of >> peak-usage-per-thread times 8 times the number of hardware threads >> supported by the system. Setting MALLOC_ARENA_MAX to a small value >> counteracts that, so it's very simple to experiment with it if you have >> a working reproducer. > > "Small value" being something like 2? Yes, that would be a good start. But my Emacs process isn't affected by this, so this setting wouldn't help there. Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill