From: Konstantin Kharlamov <hi-angel@yandex.ru>
To: Stefan Monnier <monnier@iro.umontreal.ca>
Cc: carlos@redhat.com, fweimer@redhat.com, dj@redhat.com,
45200@debbugs.gnu.org
Subject: bug#45200: [PATCH] Force Glibc to free the memory freed
Date: Wed, 03 Feb 2021 19:35:08 +0300 [thread overview]
Message-ID: <6b1ddaa05952c3d942ca74f187427c2319605621.camel@yandex.ru> (raw)
In-Reply-To: <jwvh7mti2ib.fsf-monnier+emacs@gnu.org>
On Wed, 2021-02-03 at 11:02 -0500, Stefan Monnier wrote:
> > To answer you question in another email about memory benefits given default
> > Emacs settings: well, today I tried creating a testcase that would reproduce
> > problem with default settings, but haven't really succeeded at it. I have
> > a testcase where Emacs without the patch has ≈20M more memory than the one
> > with the patch, but that's pretty small difference, and offhand I didn't
> > manage to get it increased any further.
>
> Thanks. At least that seems to indicate that glibc does its job
> properly for the way we normally use it.
>
> > So, I'm thinking of wiring the functional of memory trimming to on-idle
> > hook, without possibility to disable it.
>
> That seems hard to do (luckily), since AFAICT idle hooks only exist via
> `run-with-idle-timer` and those can always be disabled with things like
> `cancel-timer`.
Ah, okay then. I just didn't know how on-idle code should be implemented; if it would
be just making a ELisp wrapper and adding it to the hook, which people can then
remove if they want to, that's fine. I just thought there's some additional work
required to make it "disableable", I'm not opposed to the idea of it being so per se.
> > Given how small performance impact in this case would be, I see no
> > reason to even implement an option to disable it.
> > Thoughts?
>
> My main thought is that if `malloc_trim` indeed makes a significant
> difference, it's a sign that Emacs's own code did its job (it called
> `free` as it should) and that the problem is in how glibc decided not
> to return the memory to the OS.
>
> That's a behavior that can (and will) change over time outside of
> our control. So calling `malloc_trim` every time I stop typing for 10s,
> just on account of "maybe glibc didn't reclaim quite as much memory as
> we'd like this time" doesn't sound very appealing to me.
>
> It sounds like an ad-hoc quick hack. I love being able to use ad-hoc
> quick hacks, but I don't like enabling such things by default when the
> only use-cases where they're known to be useful are fairly specialized
> (and discouraged by Emacs maintainers ;-)
I totally agree with you, and I too consider it a hack.
> So I think we need more info: do the glibc maintainers consider it
> normal for glibc to behave this way? Why does it behave this way?
Very good question! I hope Glibc mainainers that are on CC list will be able to
answer. Even though I created a report on Glibc just some months ago, the problem per
se existed for a long time. And I've seen Carlos O'Donell leaving a small comment on a
similar issue with Ruby 2 years ago, which implies they're aware of this situation.
> Would the equivalent of `malloc_trim` happen anyway "at some point in
> the future"? E.g. If you create a test case where you disable GC, let
> the memory use grow to 1GB, then reset the GC vars to their default and
> keep using Emacs modestly, will the memory ever be returned to the OS or
> is an explicit call to `malloc_trim` really indispensable?
No, the memory will never be returned to OS. I can tell that right away, because the
only difference would be `free()` getting called more often.
I think it is worth mentioning here that Glibc usually does return memory to the OS
without any need in malloc_trim(0). What happens in affected applications (such as
here) is that an application stumbles upon a very special allocation pattern, which
kinda breaks Glibc algorithms of returning memory.
That's btw to the point of whether this GLibc behavior is a bug: well, sometimes it
works, sometimes it doesn't — doesn't look like purposeful behavior to me :)
> But until we get all the answers to these questions, we can already
> install the code that exposes `malloc_trim` to ELisp.
>
>
> Stefan
>
next prev parent reply other threads:[~2021-02-03 16:35 UTC|newest]
Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-12 18:43 bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Konstantin Kharlamov
2020-12-12 20:15 ` Eli Zaretskii
2020-12-12 22:44 ` Konstantin Kharlamov
2020-12-12 22:59 ` Lars Ingebrigtsen
2020-12-13 6:08 ` Eli Zaretskii
2020-12-13 5:53 ` Eli Zaretskii
2020-12-13 12:07 ` Konstantin Kharlamov
2021-01-24 15:24 ` bug#45200: [PATCH] Force Glibc to free the memory freed Konstantin Kharlamov
2021-01-24 15:40 ` Eli Zaretskii
2021-01-25 22:17 ` DJ Delorie
2021-01-25 22:28 ` Konstantin Kharlamov
2021-01-26 14:55 ` Eli Zaretskii
2021-01-26 15:02 ` Konstantin Kharlamov
2021-01-26 15:30 ` Stefan Monnier
2021-02-02 21:17 ` Konstantin Kharlamov
2021-02-03 4:45 ` Stefan Monnier
2021-02-03 4:50 ` Stefan Monnier
2021-02-03 6:04 ` Konstantin Kharlamov
2021-02-03 7:07 ` Eli Zaretskii
2021-02-03 7:15 ` Konstantin Kharlamov
2021-02-03 7:39 ` martin rudalics
2021-02-03 8:23 ` Konstantin Kharlamov
2021-02-03 9:35 ` martin rudalics
2021-02-03 9:49 ` Konstantin Kharlamov
2021-02-03 10:35 ` Konstantin Kharlamov
2021-02-03 11:06 ` martin rudalics
2021-02-03 11:08 ` Konstantin Kharlamov
2021-02-03 11:16 ` Konstantin Kharlamov
2021-02-03 12:56 ` martin rudalics
2021-02-03 13:00 ` Konstantin Kharlamov
2021-02-03 15:14 ` martin rudalics
2021-02-03 15:15 ` Stefan Monnier
2021-02-03 15:29 ` Konstantin Kharlamov
2021-02-03 16:02 ` Stefan Monnier
2021-02-03 16:35 ` Konstantin Kharlamov [this message]
2021-02-03 16:51 ` Stefan Monnier
2021-02-03 19:30 ` DJ Delorie
2021-02-03 19:36 ` DJ Delorie
2021-02-03 20:28 ` Konstantin Kharlamov
2021-02-03 20:51 ` DJ Delorie
2021-05-18 20:12 ` Konstantin Kharlamov
2021-05-19 4:11 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-05-19 4:26 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-05-19 6:46 ` Konstantin Kharlamov
2021-05-19 9:47 ` Eli Zaretskii
2021-05-19 9:55 ` Konstantin Kharlamov
2021-05-19 10:09 ` Eli Zaretskii
2021-02-03 14:51 ` Eli Zaretskii
2021-02-03 15:01 ` Konstantin Kharlamov
2021-02-03 14:44 ` Eli Zaretskii
2021-02-03 15:12 ` Andreas Schwab
2021-02-03 19:25 ` DJ Delorie
2021-02-03 19:49 ` Eli Zaretskii
2021-02-03 21:00 ` DJ Delorie
2021-02-03 20:24 ` Stefan Monnier
2021-02-03 20:42 ` DJ Delorie
2021-02-03 22:07 ` Stefan Monnier
2021-02-03 22:21 ` DJ Delorie
2021-02-03 23:32 ` Stefan Monnier
2021-02-04 0:31 ` DJ Delorie
2021-02-04 3:26 ` Stefan Monnier
2021-02-04 3:38 ` DJ Delorie
2021-02-04 3:55 ` Stefan Monnier
2021-02-04 4:02 ` DJ Delorie
2021-02-04 4:19 ` Stefan Monnier
2021-02-04 4:26 ` DJ Delorie
2021-02-04 4:04 ` DJ Delorie
2021-02-03 15:15 ` Stefan Monnier
2021-01-26 14:49 ` Eli Zaretskii
2021-01-26 16:13 ` DJ Delorie
2021-12-04 23:20 ` bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Lars Ingebrigtsen
2021-12-05 6:23 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-12-05 8:26 ` Eli Zaretskii
2022-05-01 9:43 ` bug#45200: Wishlist: There should be a `malloc-trim' function Lars Ingebrigtsen
2021-12-05 19:59 ` bug#45200: Memory leaks: (garbage-collect) fails to reclaim memory Lars Ingebrigtsen
2021-12-05 7:07 ` Eli Zaretskii
2021-01-24 18:51 ` Stefan Monnier
2021-01-24 19:00 ` Konstantin Kharlamov
2021-01-24 19:06 ` Konstantin Kharlamov
2021-01-24 19:55 ` Eli Zaretskii
2021-01-24 19:12 ` Stefan Monnier
2021-01-24 20:00 ` Konstantin Kharlamov
2021-01-24 20:11 ` Eli Zaretskii
2021-01-24 20:21 ` Konstantin Kharlamov
2021-01-24 21:20 ` Stefan Monnier
2021-01-24 21:26 ` Konstantin Kharlamov
2021-01-24 21:41 ` Stefan Monnier
2021-01-24 21:55 ` Konstantin Kharlamov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6b1ddaa05952c3d942ca74f187427c2319605621.camel@yandex.ru \
--to=hi-angel@yandex.ru \
--cc=45200@debbugs.gnu.org \
--cc=carlos@redhat.com \
--cc=dj@redhat.com \
--cc=fweimer@redhat.com \
--cc=monnier@iro.umontreal.ca \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).