From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.bugs Subject: bug#38345: 27.0.50; Permanent increase in memory consumption after opening images (or pdfs) Date: Mon, 16 Dec 2019 14:18:58 +0800 Message-ID: <87fthkokrx.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> References: <87sgme1ww7.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <83o8x0rl6d.fsf@gnu.org> <87lfs2mzo8.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <87lfs19shb.fsf@mail.linkov.net> <87a78gn5k5.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <83sgm8ox3g.fsf@gnu.org> <87zhgflxmc.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <83lfrzq1s1.fsf@gnu.org> <87d0d7w3tt.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <83zhgaloc5.fsf@gnu.org> <87zhg7jmjg.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> <83h82ej03y.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="250211"; mail-complaints-to="usenet@blaine.gmane.org" Cc: 38345@debbugs.gnu.org, juri@linkov.net To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Dec 16 07:22:17 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1igjln-0012yk-U1 for geb-bug-gnu-emacs@m.gmane.org; Mon, 16 Dec 2019 07:22:16 +0100 Original-Received: from localhost ([::1]:46668 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1igjlm-0002Dy-Jm for geb-bug-gnu-emacs@m.gmane.org; Mon, 16 Dec 2019 01:22:14 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:53334) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1igjlc-0002Dp-3m for bug-gnu-emacs@gnu.org; Mon, 16 Dec 2019 01:22:05 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1igjla-0008KX-70 for bug-gnu-emacs@gnu.org; Mon, 16 Dec 2019 01:22:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:59907) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1igjlZ-0008KG-Rz for bug-gnu-emacs@gnu.org; Mon, 16 Dec 2019 01:22:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1igjlZ-0007WY-Ok for bug-gnu-emacs@gnu.org; Mon, 16 Dec 2019 01:22:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Ihor Radchenko Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 16 Dec 2019 06:22:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38345 X-GNU-PR-Package: emacs Original-Received: via spool by 38345-submit@debbugs.gnu.org id=B38345.157647727028856 (code B ref 38345); Mon, 16 Dec 2019 06:22:01 +0000 Original-Received: (at 38345) by debbugs.gnu.org; 16 Dec 2019 06:21:10 +0000 Original-Received: from localhost ([127.0.0.1]:37647 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1igjkj-0007VL-Sp for submit@debbugs.gnu.org; Mon, 16 Dec 2019 01:21:10 -0500 Original-Received: from mail-yw1-f54.google.com ([209.85.161.54]:42207) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1igjke-0007UT-Mr for 38345@debbugs.gnu.org; Mon, 16 Dec 2019 01:21:08 -0500 Original-Received: by mail-yw1-f54.google.com with SMTP id x138so69332ywd.9 for <38345@debbugs.gnu.org>; Sun, 15 Dec 2019 22:21:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=las6eszDtNo3RaaDdNspVmjaNHstOsJJELygsZLp6ps=; b=fwZUCTyGWhKXPAFuBthaW2Zvn78xGd+RbSkOn9cUgG+5nTLjkfKyMG15PFT/gQPshj 95RbOQMiSVHBLaCuo2kk926GaYaP+owmBSuNPU1uhM4UrsxegcnHAlGZAfBIkRaMUC45 LbnS6oyGIKCCscwMW8+RHWImht++HsKzlphbo+3aaB0+tOYU9RA+d74tkord8x4cfQdI RnIEVDHx0wn2LXSKHpI2FsMj/iKG0hT6KkRuSRQb5zpM7nabTCzrgHTUZkKnoVzrcpHQ bJjOQbmIegB899QZjv8LW8VMjB5wVW5dt6kt3alSr9F2wfVDFpy+Ce5vMc/apFGptLzu HtpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=las6eszDtNo3RaaDdNspVmjaNHstOsJJELygsZLp6ps=; b=BF6fbpRHW2kfljmcXMcYc9JJSRS9kySZZWBIKOjlgqtGQXRGQYeW8cgti9J2Lrftvk mlxJc3S4t4Mhs5YtX92aLeVDp2SgFntPeSIAbFVdr7ALwWpK+qhQfwdMb70cgGti7Gix iE0duQz3DpkyjmK8yVfnouloIilhjErZgCBjnA/oHdcL8H7mfIVab5VPYX7NSPXgMyxp Q37efzziJPdHiVnZ6lmTj6BUmJXD/sy8bwm34N3mTvqgh8qwotUBCGT+ovUWyZRINClA 7jdQMWwmFzFtmPs9mIyUb6inxIud0pZdDTBdtgaKGbowr1TWP27bKRjuuHtlXj9SFeup fLog== X-Gm-Message-State: APjAAAUoUPN7IXbXwtheB+q3KmZ+O7laVVQRMdmv7nrOYt0vVHFFkOdj n8Y5mzQzFa5nMXkQxBLbLXY= X-Google-Smtp-Source: APXvYqwiJAt3ApLlO05xPmM4CXrRi6AUIwhqQOBgRmcAgSaHjtWj72jeiyutXgmlESjNPiDd529c3w== X-Received: by 2002:a81:1d4f:: with SMTP id d76mr16395178ywd.200.1576477259060; Sun, 15 Dec 2019 22:20:59 -0800 (PST) Original-Received: from localhost ([5.226.137.4]) by smtp.gmail.com with ESMTPSA id l204sm8234886ywb.79.2019.12.15.22.20.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2019 22:20:58 -0800 (PST) In-Reply-To: <83h82ej03y.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:173416 Archived-At: > What you describe is the situation where Emacs enlarges the image > cache because the existing cache can no longer hold all the images we > currently know about. This is normal, and cannot cause a leak. Agree. In my earlier email, it described my trial to expunge the images from cache more aggressively: >> also, I repeated my tests manually calling image-clear-cache every Nth >> image. Every invocation of image-clear-cache does reduce the memory >> consumption, but there is always some residual remaining unrealeased >> (see the attached images). The residual seems to scale with the number >> of images in the cache before clearing. You suggested to look at the C code. However, looking at the C code, I don't understand then why the memory consumption is increasing. I can only see that too many images in c->images array can cause extra memory consumption, which cannot cause the observed memory leak. Do you have any idea why even aggressive cache clearing causes memory consumption increase? Best, Ihor Eli Zaretskii writes: >> From: Ihor Radchenko >> Cc: juri@linkov.net, 38345@debbugs.gnu.org >> Date: Thu, 05 Dec 2019 14:48:03 +0800 >> >> For the image cache data structure, I can only see one place where it >> may request extra memory allocation. It is when the number of images in >> the cache exceeds the size of c->images array (= c->size, which is 50 by >> default). I observed memory consumption increase even when the frequency >> of cache clearing in my test was <50, which makes it unclear for me >> where the extra memory consumption is coming from. > > What you describe is the situation where Emacs enlarges the image > cache because the existing cache can no longer hold all the images we > currently know about. This is normal, and cannot cause a leak. > > The suggestion in this discussion was to expunge images from the cache > more aggressively, and I thought you had difficulties in understand > how this is done currently. > >> > I don't think replacing the system malloc on GNU/Linux systems (let >> > alone on others) is an idea we'd like to pursue. You may have more >> > luck playing with mallopt calls in init_alloc_once_for_pdumper >> > (assuming your build defines DOUG_LEA_MALLOC). >> >> Do I understand correctly that you refer to emacs compiled with >> alternative Doug Lea's implementation of malloc? > > No. AFAIK, any compilation on GNU/Linus defines DOUG_LEA_MALLOC. > >> Actually, I tried to find a way to compile emacs with alternative >> variants of malloc, but I did not find how to do it. > > I don't think we support it, at least not easily. > >> P.S. I am running emacs with jemalloc for a few days and the overall >> impression is that emacs became a lot more responsive. Previously, I >> had a slow overtime degradation of delay between commands as emacs >> process ran for a long time, up to the point when I can type a whole >> sentence until emacs finally displays it on screen. Now, I do not see so >> much degradation. > > This has got to be a separate issue, perhaps related to GC (do you > have customizations of any GC-related variables?). It should be > reported and analyzed separately, as it is unrelated to image caching.