bug#45200: [PATCH] Force Glibc to free the memory freed

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> So I think we need more info: do the glibc maintainers consider it
>> normal for glibc to behave this way?  Why does it behave this way?
> Very good question! I hope Glibc mainainers that are on CC list will be able to
> answer.  Even though I created a report on Glibc just some months ago, the problem per
> se existed for a long time. And I've seen Carlos O'Donell leaving a small comment on a
> similar issue with Ruby 2 years ago, which implies they're aware of this situation.

I mean, I understand why glibc doesn't always immediately release memory
to the OS, but I'd like to better understand whether we can expect that
there is a mechanism in glibc to avoid it keeping 500MB of free memory
around eternally when the subsequent actual use remains below 100MB.

>> Would the equivalent of `malloc_trim` happen anyway "at some point in
>> the future"?  E.g. If you create a test case where you disable GC, let
>> the memory use grow to 1GB, then reset the GC vars to their default and
>> keep using Emacs modestly, will the memory ever be returned to the OS or
>> is an explicit call to `malloc_trim` really indispensable?
>
> No, the memory will never be returned to OS. I can tell that right
> away, because the only difference would be `free()` getting called
> more often.

I'm not sure I follow: if I were to implement a malloc library, calls to
`free` would be exactly the place where I'd decide that some memory
should be returned to the OS, so even if a given call to `free` might
not return the now-unused 400MB, I'd expect that this memory might be
finally returned after some number of other/unrelated calls to `free`.

> I think it is worth mentioning here that Glibc usually does return
> memory to the OS without any need in malloc_trim(0). What happens in
> affected applications (such as here) is that an application stumbles
> upon a very special allocation pattern, which kinda breaks Glibc
> algorithms of returning memory.

Indeed.  The question is whether it's a bug in glibc or whether it's
something that the upper layer should deal with.  Knowing a bit better
why/when this happens would also help the upper layer decide when to
call `malloc_trim` if indeed that's needed (another useful info would be
a function that returns the mount of memory there is to trim, assuming
that info can be obtained much more cheaply than a call to
`malloc_trim`).


        Stefan