From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Florian Weimer Newsgroups: gmane.emacs.bugs Subject: bug#43389: 28.0.50; Emacs memory leaks Date: Mon, 16 Nov 2020 21:42:39 +0100 Message-ID: <87wnylm3sw.fsf@oldenburg2.str.redhat.com> References: <87r1r5428d.fsf@web.de> <874kmcvlbj.fsf@mail.trevorbentley.com> <83imasb0te.fsf@gnu.org> <871rgzvbme.fsf@mail.trevorbentley.com> <83lff6zm8f.fsf@gnu.org> <838sb1rrar.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="17682"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Cc: carlos@redhat.com, 43389@debbugs.gnu.org, dj@redhat.com To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Nov 16 21:43:34 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kelLZ-0004SY-Rf for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 16 Nov 2020 21:43:33 +0100 Original-Received: from localhost ([::1]:52746 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kelLU-0006fc-UE for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 16 Nov 2020 15:43:28 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:45394) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kelLB-0006fC-5u for bug-gnu-emacs@gnu.org; Mon, 16 Nov 2020 15:43:10 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:45923) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kelL4-0002pk-Ih for bug-gnu-emacs@gnu.org; Mon, 16 Nov 2020 15:43:07 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kelL4-000163-FI for bug-gnu-emacs@gnu.org; Mon, 16 Nov 2020 15:43:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Florian Weimer Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 16 Nov 2020 20:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43389 X-GNU-PR-Package: emacs Original-Received: via spool by 43389-submit@debbugs.gnu.org id=B43389.16055593724198 (code B ref 43389); Mon, 16 Nov 2020 20:43:02 +0000 Original-Received: (at 43389) by debbugs.gnu.org; 16 Nov 2020 20:42:52 +0000 Original-Received: from localhost ([127.0.0.1]:57469 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kelKt-00015d-TQ for submit@debbugs.gnu.org; Mon, 16 Nov 2020 15:42:52 -0500 Original-Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47710) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kelKr-00015V-52 for 43389@debbugs.gnu.org; Mon, 16 Nov 2020 15:42:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605559368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=d8cLFNW8Tl6xYnQQcNYyT7LlqOLx5lsjYmTAh9zXTbc=; b=BJ/IgEg2LZ3DtYurQyYGSItZILUYUXKGxWkR2yUQbrFL4YQcXlSrF4nEekSL6JS8oM1BVI PAZUPctGMjItA+J4qAj3gwtczUF3qAZZDytiiIV/dWN6Y1Zj9HGYTSlV53qY00ZVyr3ngZ c6EP5pxNyHTa3ktC9TTFp1odWYl29c4= Original-Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-439-YaULm9ewPZC_qToDRn-mtQ-1; Mon, 16 Nov 2020 15:42:46 -0500 X-MC-Unique: YaULm9ewPZC_qToDRn-mtQ-1 Original-Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8F0241007B26; Mon, 16 Nov 2020 20:42:45 +0000 (UTC) Original-Received: from oldenburg2.str.redhat.com (ovpn-113-222.ams2.redhat.com [10.36.113.222]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1AD2D5C1D7; Mon, 16 Nov 2020 20:42:40 +0000 (UTC) In-Reply-To: <838sb1rrar.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 16 Nov 2020 22:16:12 +0200") X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=fweimer@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:193473 Archived-At: * Eli Zaretskii: > For some time (several months, I think) we have reports from Emacs > users that the memory footprints of their Emacs sessions sometimes > start growing very quickly, from several hundreds of MBytes to several > gigabytes in a day or even just few hours, and in some cases causing > the OOMK to kick in and kill the Emacs process. The last time I saw this was a genuine memory leak in the Emacs C code. Just saying. 8-) > The questions that I'd like to eventually be able to answer are: > > . is this indeed due to some malloc'ed chunk that is being used for > prolonged periods of time, and prevents releasing parts of the > heap to the system? IOW, is this pathological, but correct > behavior, or is this some bug? > > . if this is correct behavior, can Emacs do something to avoid > triggering it? For example, should we consider tuning glibc's > malloc in some way, by changing the 3 calls to mallopt in > init_alloc_once_for_pdumper? > > Your thoughts and help in investigating these problems will be highly > appreciated. Please feel free to ask any questions you come up with, > including about the details of Emacs's memory management and anything > related. There is an issue with reusing posix_memalign allocations. On my system (running Emacs 27.1 as supplied by Fedora 32), I only see such allocations as the backing storage for the glib (sic) slab allocator. It gets exercised mostly when creating UI elements, as far as I can tell. In theory, these backing allocations should be really long-term and somewhat limited, so the fragmentation peculiar to aligned allocations issue should not be a concern. There is actually a glibc patch floating around that fixes the aligned allocation problem, at some (hopefully limited) performance cost to aligned allocations. We want to get it reviewed and integrated into upstream glibc. If there is a working reproducer, we could run it against a patched glibc. The other issue we have is that thread counts has exceeded in recent times more than system memory, and glibc basically scales RSS overhead with thread count, not memory. A use of libgomp suggests that many threads might indeed be spawned. If their lifetimes overlap, it would not be unheard of to end up with some RSS overhead in the order of peak-usage-per-thread times 8 times the number of hardware threads supported by the system. Setting MALLOC_ARENA_MAX to a small value counteracts that, so it's very simple to experiment with it if you have a working reproducer. Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill