bug#38345: 27.0.50; Permanent increase in memory consumption after opening images (or pdfs)

[Ihor Radchenko <yantar92@gmail.com>]

> What you describe is the situation where Emacs enlarges the image
> cache because the existing cache can no longer hold all the images we
> currently know about.  This is normal, and cannot cause a leak.

Agree. 

In my earlier email, it described my trial to expunge the images from
cache more aggressively:

>> also, I repeated my tests manually calling image-clear-cache every Nth
>> image. Every invocation of image-clear-cache does reduce the memory
>> consumption, but there is always some residual remaining unrealeased
>> (see the attached images). The residual seems to scale with the number
>> of images in the cache before clearing.

You suggested to look at the C code.

However, looking at the C code, I don't understand then why the
memory consumption is increasing. I can only see that too many images in
c->images array can cause extra memory consumption, which cannot cause
the observed memory leak.

Do you have any idea why even aggressive cache clearing causes memory
consumption increase?

Best,
Ihor


Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@gmail.com>
>> Cc: juri@linkov.net, 38345@debbugs.gnu.org
>> Date: Thu, 05 Dec 2019 14:48:03 +0800
>> 
>> For the image cache data structure, I can only see one place where it
>> may request extra memory allocation. It is when the number of images in
>> the cache exceeds the size of c->images array (= c->size, which is 50 by
>> default). I observed memory consumption increase even when the frequency
>> of cache clearing in my test was <50, which makes it unclear for me
>> where the extra memory consumption is coming from.
>
> What you describe is the situation where Emacs enlarges the image
> cache because the existing cache can no longer hold all the images we
> currently know about.  This is normal, and cannot cause a leak.
>
> The suggestion in this discussion was to expunge images from the cache
> more aggressively, and I thought you had difficulties in understand
> how this is done currently.
>
>> > I don't think replacing the system malloc on GNU/Linux systems (let
>> > alone on others) is an idea we'd like to pursue.  You may have more
>> > luck playing with mallopt calls in init_alloc_once_for_pdumper
>> > (assuming your build defines DOUG_LEA_MALLOC).
>> 
>> Do I understand correctly that you refer to emacs compiled with
>> alternative Doug Lea's implementation of malloc?
>
> No.  AFAIK, any compilation on GNU/Linus defines DOUG_LEA_MALLOC.
>
>> Actually, I tried to find a way to compile emacs with alternative
>> variants of malloc, but I did not find how to do it. 
>
> I don't think we support it, at least not easily.
>
>> P.S. I am running emacs with jemalloc for a few days and the overall
>> impression is that emacs became a lot more responsive. Previously, I
>> had a slow overtime degradation of delay between commands as emacs
>> process ran for a long time, up to the point when I can type a whole
>> sentence until emacs finally displays it on screen. Now, I do not see so
>> much degradation. 
>
> This has got to be a separate issue, perhaps related to GC (do you
> have customizations of any GC-related variables?).  It should be
> reported and analyzed separately, as it is unrelated to image caching.