From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Trevor Bentley Newsgroups: gmane.emacs.bugs Subject: bug#43389: 28.0.50; Emacs memory leaks using hard disk all time Date: Thu, 26 Nov 2020 13:37:54 +0100 Message-ID: <87a6v4thst.fsf@mail.trevorbentley.com> References: <86y2j2brg2.fsf@protected.rcdrun.com> <83blfxth7c.fsf@gnu.org> <83y2j0qb2v.fsf@gnu.org> <831rgppg3w.fsf@gnu.org> <83zh3czbvz.fsf@gnu.org> <83blfovzxz.fsf@gnu.org> <87o8jnu5f2.fsf@mail.trevorbentley.com> <83o8jmu49z.fsf@gnu.org> <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; format=flowed Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10057"; mail-complaints-to="usenet@ciao.gmane.io" Cc: fweimer@redhat.com, 43389@debbugs.gnu.org, bugs@gnu.support, dj@redhat.com, michael_heerdegen@web.de, To: Carlos O'Donell , Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Nov 26 13:40:17 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kiGZL-0002TP-K9 for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 26 Nov 2020 13:40:16 +0100 Original-Received: from localhost ([::1]:59464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiGZK-0004rv-8P for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 26 Nov 2020 07:40:14 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59076) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiGYB-0004rV-Qf for bug-gnu-emacs@gnu.org; Thu, 26 Nov 2020 07:39:04 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:56695) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kiGYA-0003uS-SC for bug-gnu-emacs@gnu.org; Thu, 26 Nov 2020 07:39:03 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kiGYA-0001Lr-Pt for bug-gnu-emacs@gnu.org; Thu, 26 Nov 2020 07:39:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Trevor Bentley Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 26 Nov 2020 12:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43389 X-GNU-PR-Package: emacs Original-Received: via spool by 43389-submit@debbugs.gnu.org id=B43389.16063942855106 (code B ref 43389); Thu, 26 Nov 2020 12:39:02 +0000 Original-Received: (at 43389) by debbugs.gnu.org; 26 Nov 2020 12:38:05 +0000 Original-Received: from localhost ([127.0.0.1]:40004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiGXF-0001KI-Ae for submit@debbugs.gnu.org; Thu, 26 Nov 2020 07:38:05 -0500 Original-Received: from mail.trevorbentley.com ([37.187.5.80]:35451) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiGXA-0001JY-VK for 43389@debbugs.gnu.org; Thu, 26 Nov 2020 07:38:02 -0500 Original-Received: from localhost (c188-150-0-48.bredband.comhem.se [188.150.0.48]) by mail.trevorbentley.com (Postfix) with ESMTPSA id C2A0A60DA4; Thu, 26 Nov 2020 13:37:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mail.trevorbentley.com; s=mail; t=1606394274; bh=mGf1Bzdq1CN9mGCzRuOrbQ+0XC/sZ9PQj+DMkY22BZg=; h=From:To:Cc:Cc:Subject:In-Reply-To:References:Date:From; b=At74EIb0sWDhlaSoKr+5haezWAQm/NTY/X6vJhlwkgK80gfnVF9f2cJDMT+F61IMV BTbHpcZ/v9FI/0MMCGfNb3wE/mCRzII5LgHhSTgxiuzlm+y6bYQ1TDwuqUYGQTJ9X+ guvpgM6jVPBrHBAi95KRoWtVdzjLR4FooY86F8XM= In-Reply-To: <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:194298 Archived-At: > You want visibility into what is USING that memory. > > With glibc-malloc-trace-utils you can try to do that with: > > LD_PRELOAD=libmtrace.so \ MTRACE_CTL_FILE=/home/user/app.mtr \ > MTRACE_CTL_BACKTRACE=1 \ ./app > > This will use libgcc's unwinder to get a copy of the malloc > caller address and then we'll have to decode that based on a > /proc/self/maps. > > Next steps: - Get a glibc-malloc-trace-utils trace of the > application ratcheting. - Get a copy of /proc/$PID/maps for the > application (shorter version of smaps). > Oh, this is going to be a problem. I guess it is producing one trace file per thread? I ran it with libmtrace overnight. Memory usage was very high, but it doesn't look like the same problem. I hit 1550MB of RSS, but smaps reported only ~350MB of that was in the heap, which seemed reasonable for the ~150MB that emacs reported it was using. Does libmtrace add a lot of memory overhead? However, libmtrace has made 4968 files totalling 26GB in that time. Ouch. It's going to be hard to tell when I hit the bug under libmtrace, questionable whether the report will even fit on my disk, and tricky to share however many tens of gigabytes of trace files it results in. If it's one trace per thread, though, then we at least know that my emacs process in question is blazing through threads. That could be relevant. Other thing to note (for Eli): I wrapped garbage-collect like so: --- (defun trev/garbage-collect (orig-fun &rest args) (message "%s -- Starting garbage-collect." (current-time-string)) (let ((time (current-time)) (result (apply orig-fun args))) (message "%s -- Finished garbage-collect in %.06f" (current-time-string) (float-time (time-since time))) result)) (add-function :around (symbol-function 'garbage-collect) #'trev/garbage-collect) --- This printed a start and stop message each time I evaluated garbage-collect manually. It did not print any messages in 11 hours of running unattended. This is with an active network connection receiving messages fairly frequently, so there was plenty of consing going on. Hard for me to judge if it should run any garbage collection in that time, but I would have expected so. -Trevor