From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Konstantin Kharlamov Newsgroups: gmane.emacs.bugs Subject: bug#45200: [PATCH] Force Glibc to free the memory freed Date: Tue, 18 May 2021 23:12:53 +0300 Message-ID: References: <83k0rz21dw.fsf@gnu.org> <331805c74fc5d3d412dd2065030b11fa3343710d.camel@yandex.ru> <8a91fc16f93298bca1281b43d6821ae3621376dc.camel@yandex.ru> <7ffacc5f-fc0e-a5f8-104d-2c0ae8e06953@gmx.at> <3f4f8b3034e9f52f48ec520f2732b1687bb24dbc.camel@yandex.ru> <31608795-6155-c7c9-7d94-6024adb0a3b9@gmx.at> <09cc70f14e90f6b13b51b0536fae76e87dfe3f42.camel@yandex.ru> <55be0318-c907-bf9b-d644-d88abb816871@gmx.at> <35163027-a5b7-4a6c-6700-69d34fecf451@gmx.at> <824625557ce288b0cbc3edd66ff730afd479b511.camel@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33554"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Evolution 3.40.1 Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com, 45200@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue May 18 22:28:46 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lj6Ka-0008Z8-U1 for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 18 May 2021 22:28:45 +0200 Original-Received: from localhost ([::1]:53034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lj6KZ-0004iX-Vq for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 18 May 2021 16:28:44 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43402) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lj66M-00056y-3H for bug-gnu-emacs@gnu.org; Tue, 18 May 2021 16:14:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:45838) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lj66L-0007Tj-PV for bug-gnu-emacs@gnu.org; Tue, 18 May 2021 16:14:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lj66L-0008Sr-LV for bug-gnu-emacs@gnu.org; Tue, 18 May 2021 16:14:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Konstantin Kharlamov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 18 May 2021 20:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45200 X-GNU-PR-Package: emacs Original-Received: via spool by 45200-submit@debbugs.gnu.org id=B45200.162136878432464 (code B ref 45200); Tue, 18 May 2021 20:14:01 +0000 Original-Received: (at 45200) by debbugs.gnu.org; 18 May 2021 20:13:04 +0000 Original-Received: from localhost ([127.0.0.1]:57384 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lj65Q-0008RY-95 for submit@debbugs.gnu.org; Tue, 18 May 2021 16:13:04 -0400 Original-Received: from forward106o.mail.yandex.net ([37.140.190.187]:59656) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lj65N-0008Qa-6U for 45200@debbugs.gnu.org; Tue, 18 May 2021 16:13:03 -0400 Original-Received: from sas1-de9e0dd89469.qloud-c.yandex.net (sas1-de9e0dd89469.qloud-c.yandex.net [IPv6:2a02:6b8:c14:3997:0:640:de9e:dd8]) by forward106o.mail.yandex.net (Yandex) with ESMTP id 2F5945060765; Tue, 18 May 2021 23:12:54 +0300 (MSK) Original-Received: from sas1-27140bb19246.qloud-c.yandex.net (sas1-27140bb19246.qloud-c.yandex.net [2a02:6b8:c08:1803:0:640:2714:bb1]) by sas1-de9e0dd89469.qloud-c.yandex.net (mxback/Yandex) with ESMTP id me84m2Lgkc-CrKiLuZP; Tue, 18 May 2021 23:12:54 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1621368774; bh=u5r/UPXFKIwXeCWWsxWLy+cP3Gl9q8vZ4QltKGsp5xs=; h=In-Reply-To:Cc:To:From:Subject:Message-ID:References:Date; b=p1xNqUNKJLdkzxCk5KLnHawRvwI3/BS0Oy/w84hKI6kC1bOWa0ZzSZ3yLLgnYstEi jTQH14QEU56eesgRCgmOICCUUW3eS+Dbv0Ot82CEx7T3BVwlSAw9yPhIFv5VGzpffk a8Jd+GDAtwj0m0sgzcp80cJSKOqBJJmbrLY5Sd2w= Authentication-Results: sas1-de9e0dd89469.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Original-Received: by sas1-27140bb19246.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id Q7m2S8yAr8-CrLqMBGm; Tue, 18 May 2021 23:12:53 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:206835 Archived-At: Okay, apparently I'm not the only one affected by the memory problem¹, so I'll try to make the necessary change for it to go over the finish line. I have a question though. So, wrapping `malloc_trim` in elisp and adding it to idle with `(run-with-idle-timer)` sounds simple. But, where in code should I do the `(run-with-idle-timer)` call, i.e. so it's called as is emacs is getting initialized? 1: https://www.reddit.com/r/emacs/comments/n13v5l/what_is_the_next_big_feature_after_native_comp/gwc5g77/?context=3 On Wed, 2021-02-03 at 11:02 -0500, Stefan Monnier wrote: > > To answer you question in another email about memory benefits given default > > Emacs settings: well, today I tried creating a testcase that would reproduce > > problem with default settings, but haven't really succeeded at it.  I have > > a testcase where Emacs without the patch has ≈20M more memory than the one > > with the patch, but that's pretty small difference, and offhand I didn't > > manage to get it increased any further. > > Thanks.  At least that seems to indicate that glibc does its job > properly for the way we normally use it. > > > So, I'm thinking of wiring the functional of memory trimming to on-idle > > hook, without possibility to disable it. > > That seems hard to do (luckily), since AFAICT idle hooks only exist via > `run-with-idle-timer` and those can always be disabled with things like > `cancel-timer`. > > > Given how small performance impact in this case would be, I see no > > reason to even implement an option to disable it. > > Thoughts? > > My main thought is that if `malloc_trim` indeed makes a significant > difference, it's a sign that Emacs's own code did its job (it called > `free` as it should) and that the problem is in how glibc decided not > to return the memory to the OS. > > That's a behavior that can (and will) change over time outside of > our control.  So calling `malloc_trim` every time I stop typing for 10s, > just on account of "maybe glibc didn't reclaim quite as much memory as > we'd like this time" doesn't sound very appealing to me. > > It sounds like an ad-hoc quick hack.  I love being able to use ad-hoc > quick hacks, but I don't like enabling such things by default when the > only use-cases where they're known to be useful are fairly specialized > (and discouraged by Emacs maintainers ;-) > > So I think we need more info: do the glibc maintainers consider it > normal for glibc to behave this way?  Why does it behave this way? > Would the equivalent of `malloc_trim` happen anyway "at some point in > the future"?  E.g. If you create a test case where you disable GC, let > the memory use grow to 1GB, then reset the GC vars to their default and > keep using Emacs modestly, will the memory ever be returned to the OS or > is an explicit call to `malloc_trim` really indispensable? > > But until we get all the answers to these questions, we can already > install the code that exposes `malloc_trim` to ELisp. > > >         Stefan >