From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Lars Ingebrigtsen Newsgroups: gmane.emacs.bugs Subject: bug#33275: 27.0.50; Image cache pruning Date: Mon, 05 Nov 2018 18:36:39 +0100 Message-ID: References: <83va5bcxxu.fsf@gnu.org> <83sh0fcx2r.fsf@gnu.org> <83o9b3cu4f.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1541439313 27803 195.159.176.226 (5 Nov 2018 17:35:13 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 5 Nov 2018 17:35:13 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: 33275@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Nov 05 18:35:09 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJimK-00079N-Qf for geb-bug-gnu-emacs@m.gmane.org; Mon, 05 Nov 2018 18:35:09 +0100 Original-Received: from localhost ([::1]:36633 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJioQ-0006xz-Un for geb-bug-gnu-emacs@m.gmane.org; Mon, 05 Nov 2018 12:37:18 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46190) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJioE-0006vL-Gk for bug-gnu-emacs@gnu.org; Mon, 05 Nov 2018 12:37:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJioA-0005iq-8V for bug-gnu-emacs@gnu.org; Mon, 05 Nov 2018 12:37:06 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:59505) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gJioA-0005i4-3E for bug-gnu-emacs@gnu.org; Mon, 05 Nov 2018 12:37:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gJio9-0006YB-Vm for bug-gnu-emacs@gnu.org; Mon, 05 Nov 2018 12:37:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 05 Nov 2018 17:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33275 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 33275-submit@debbugs.gnu.org id=B33275.154143940825159 (code B ref 33275); Mon, 05 Nov 2018 17:37:01 +0000 Original-Received: (at 33275) by debbugs.gnu.org; 5 Nov 2018 17:36:48 +0000 Original-Received: from localhost ([127.0.0.1]:35530 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJinw-0006Xj-Fp for submit@debbugs.gnu.org; Mon, 05 Nov 2018 12:36:48 -0500 Original-Received: from lamora.getmail.no ([84.210.184.7]:60571) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJinu-0006XW-NH for 33275@debbugs.gnu.org; Mon, 05 Nov 2018 12:36:47 -0500 Original-Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id 18CD56083C; Mon, 5 Nov 2018 18:36:40 +0100 (CET) Original-Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id OcUNj5dXngkp; Mon, 5 Nov 2018 18:36:39 +0100 (CET) Original-Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id 9507C6083E; Mon, 5 Nov 2018 18:36:39 +0100 (CET) X-Virus-Scanned: amavisd-new at lamora.get.c.bitbit.net Original-Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4mkZH3iNi6IM; Mon, 5 Nov 2018 18:36:39 +0100 (CET) Original-Received: from stories (cm-84.212.221.165.getinternet.no [84.212.221.165]) by lamora.getmail.no (Postfix) with ESMTPSA id 649906083C; Mon, 5 Nov 2018 18:36:39 +0100 (CET) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAGFBMVEW6gTlVGIFdHox+SDZD EmhXGoJRFnz8xUXqvZ6yAAACO0lEQVQ4jW2UzY+DIBDFaa3bM0a8S2rPJG7iVbvUO0Q8S4NzbtKD //4OoK2bLP1I01/ePN4wQsBo/t8iBdX/IjIyzTUH6Yquu1wenF+rWwDK6IpzJ9fFr73rV0WGlVzv 4NYNQbGWUoYpzuUPea/vewBGzYbry5Scl+d5we8ljR6mmtF9mGqRvAR5EdGc+hVYzpVr67p+hXdz vFUeaMN0VSBAxQZ4BHNmrAf4r2fNocNgHlhuQEbwRwFQMcgnEsyfQmzAWKqYzdvlvb56ju4EwDJr 5Q6gwnuABcOyy/QtMdtCPIjmwGBmWd4KQYSoSf1qTjJ4OABgmUOQkAZB0qS9YbEU2McwiXYDXz3N gjnM5q1IEJzkCizzyT8g7SieKilgBkvzCYGIAEtRHRRWKa+oxVZqxFreA8ZZtqL2wJsfu5JtwObT spxD7ldzkAVl5q1I6giei/fIqhAQGJ7HD4kdWdJebR4Acbt4JqG7JTUbQPN2A6mk2BNiLX0rsI8e YMBQCoHNA8DsCLAlJgQsEMjoQWIpFZOz0Vo3fcCxi8DSwlLnzUUEviU67spirwg5n9czxxzrdmcz tFtLfKmg8Mkt24/PSZZMmwBmjaVEnSTE9yS9lVT7Xo3A2OBz15hkNz6Yg9KhfYYOPkOpyqxgVvtJ TPtKr9t9mGH6gEPPowIKzfPdw/kVJ9EDRsfHHS7j7X65Ph7XfgNWfy4AvyKwOKJMDX3nXH7v3ICf YlMA0yX+rP5eMgEYpXc3UIWv8hctdBMsfua7RgAAAABJRU5ErkJggg== In-Reply-To: <83o9b3cu4f.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 05 Nov 2018 19:27:28 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:152066 Archived-At: Eli Zaretskii writes: > I think it's reasonable to expect 2GB worth of image files to take > several times that in memory. (Btw, visiting a 16GB file usually > takes twice that much during insert-file-contents.) It would be reasonable if the code had asked for those images to be kept in memory, but it hasn't. Emacs decides that on its own without saying that it's doing that. It'd be similarly surprising if (dolist (file (directory-files "/directory/with/many/images" t "png$")) (with-temp-buffer (insert-file-contents-literally file))) were to lead to Emacs growing uncontrollably. At present, the image cache is purely based on timestamps, if I recall correctly? If we wish to fix this unexpected behaviour, we could also look at the size of the image cache and perhaps try to limit it, or if we wanted to be really fancy, we could somehow check whether there's any references to the images anywhere (beyond the image cache itself). I think the latter would mean adding some... fun... to the gc system? Or does Emacs have weak hashes? In which case the image cache could be a weak hash and the problem would resolve itself? > I think we should try to add some code that would call > display_malloc_warning and/or memory_full, before the system runs out > of memory. That should be enough to prevent OOM in such cases. Can > you spot why we never called memory_full in this case? Sorry, I'm not familiar with the memory_full stuff... -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no