unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
From: Trevor Bentley <trevor@trevorbentley.com>
To: Carlos O'Donell <carlos@redhat.com>, Eli Zaretskii <eliz@gnu.org>
Cc: fweimer@redhat.com, 43389@debbugs.gnu.org, bugs@gnu.support,
	dj@redhat.com, michael_heerdegen@web.de,
Subject: bug#43389: 28.0.50; Emacs memory leaks using hard disk all time
Date: Thu, 26 Nov 2020 13:37:54 +0100	[thread overview]
Message-ID: <87a6v4thst.fsf@mail.trevorbentley.com> (raw)
In-Reply-To: <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com>

> You want visibility into what is USING that memory. 
> 
> With glibc-malloc-trace-utils you can try to do that with: 
> 
> LD_PRELOAD=libmtrace.so \ MTRACE_CTL_FILE=/home/user/app.mtr \ 
> MTRACE_CTL_BACKTRACE=1 \ ./app 
> 
> This will use libgcc's unwinder to get a copy of the malloc 
> caller address and then we'll have to decode that based on a 
> /proc/self/maps. 
> 
> Next steps: - Get a glibc-malloc-trace-utils trace of the 
> application ratcheting.  - Get a copy of /proc/$PID/maps for the 
> application (shorter version of smaps). 
> 

Oh, this is going to be a problem.  I guess it is producing one 
trace file per thread?

I ran it with libmtrace overnight.  Memory usage was very high, 
but it doesn't look like the same problem.  I hit 1550MB of RSS, 
but smaps reported only ~350MB of that was in the heap, which 
seemed reasonable for the ~150MB that emacs reported it was using. 
Does libmtrace add a lot of memory overhead?

However, libmtrace has made 4968 files totalling 26GB in that 
time.  Ouch.

It's going to be hard to tell when I hit the bug under libmtrace, 
questionable whether the report will even fit on my disk, and 
tricky to share however many tens of gigabytes of trace files it 
results in.

If it's one trace per thread, though, then we at least know that 
my emacs process in question is blazing through threads.  That 
could be relevant.

Other thing to note (for Eli): I wrapped garbage-collect like so:

---
(defun trev/garbage-collect (orig-fun &rest args) 
  (message "%s -- Starting garbage-collect." 
  (current-time-string)) (let ((time (current-time)) 
        (result (apply orig-fun args))) 
    (message "%s -- Finished garbage-collect in %.06f" 
    (current-time-string) (float-time (time-since time))) result)) 
(add-function :around (symbol-function 'garbage-collect) 
#'trev/garbage-collect)
---

This printed a start and stop message each time I evaluated 
garbage-collect manually.  It did not print any messages in 11 
hours of running unattended.  This is with an active network 
connection receiving messages fairly frequently, so there was 
plenty of consing going on.  Hard for me to judge if it should run 
any garbage collection in that time, but I would have expected so.

-Trevor





  parent reply	other threads:[~2020-11-26 12:37 UTC|newest]

Thread overview: 166+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-15 14:55 bug#44666: 28.0.50; malloc-info: Emacs became not responsive, using hard disk all time Jean Louis
2020-11-16 16:11 ` Eli Zaretskii
2020-11-16 16:17   ` Jean Louis
2020-11-17 15:04     ` Eli Zaretskii
2020-11-19  6:59       ` Jean Louis
2020-11-19 14:37         ` bug#43389: 28.0.50; Emacs memory leaks " Eli Zaretskii
2020-11-20  3:16           ` Jean Louis
2020-11-20  8:10             ` Eli Zaretskii
2020-11-22 19:52               ` Jean Louis
2020-11-22 20:16                 ` Eli Zaretskii
2020-11-23  3:41                   ` Carlos O'Donell
2020-11-23  8:11                   ` Jean Louis
2020-11-23  9:59                     ` Eli Zaretskii
2020-11-23 17:19                       ` Arthur Miller
2020-11-23 17:44                         ` Eli Zaretskii
2020-11-23 18:34                           ` Arthur Miller
2020-11-23 19:06                             ` Jean Louis
2020-11-23 19:15                             ` Eli Zaretskii
2020-11-23 19:49                               ` Arthur Miller
2020-11-23 20:04                                 ` Eli Zaretskii
2020-11-23 21:12                                   ` Arthur Miller
2020-11-24  2:07                                   ` Arthur Miller
2020-11-23 20:31                                 ` Jean Louis
2020-11-23 21:22                                   ` Arthur Miller
2020-11-24  5:29                                     ` Jean Louis
2020-11-24  8:15                                       ` Arthur Miller
2020-11-24  9:06                                         ` Jean Louis
2020-11-24  9:27                                           ` Arthur Miller
2020-11-24 17:18                                             ` Jean Louis
2020-11-25 14:59                                               ` Arthur Miller
2020-11-25 15:09                                                 ` Jean Louis
2020-11-23 13:27                   ` Jean Louis
2020-11-23 15:54                     ` Carlos O'Donell
2020-11-23 18:58                       ` Jean Louis
2020-11-23 19:34                         ` Eli Zaretskii
2020-11-23 19:49                           ` Jean Louis
2020-11-23 20:04                           ` Carlos O'Donell
2020-11-23 20:16                             ` Eli Zaretskii
2020-11-23 19:37                         ` Carlos O'Donell
2020-11-23 19:55                           ` Jean Louis
2020-11-23 20:06                             ` Carlos O'Donell
2020-11-23 20:18                               ` Jean Louis
2020-11-23 20:31                                 ` Eli Zaretskii
2020-11-23 20:41                                   ` Jean Louis
2020-11-23 20:53                                     ` Andreas Schwab
2020-11-23 21:09                                       ` Jean Louis
2020-11-24  3:25                                       ` Eli Zaretskii
2020-11-23 20:10                             ` Eli Zaretskii
2020-11-23 19:50                     ` Carlos O'Donell
2020-11-23 19:59                       ` Jean Louis
2020-11-23 10:59               ` Jean Louis
2020-11-23 15:46                 ` Eli Zaretskii
2020-11-23 17:29                   ` Arthur Miller
2020-11-23 17:45                     ` Eli Zaretskii
2020-11-23 18:40                       ` Arthur Miller
2020-11-23 19:23                         ` Eli Zaretskii
2020-11-23 19:38                           ` Arthur Miller
2020-11-23 19:52                             ` Eli Zaretskii
2020-11-23 20:03                               ` Arthur Miller
2020-11-23 19:39                           ` Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors
2020-11-23 19:59                             ` Arthur Miller
2020-11-23 20:15                               ` Eli Zaretskii
2020-11-23 21:15                                 ` Arthur Miller
2020-11-23 20:53                               ` Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors
2020-11-23 18:33                   ` Jean Louis
2020-11-23 21:30                   ` Trevor Bentley
2020-11-23 22:11                     ` Trevor Bentley
2020-11-24 16:07                     ` Eli Zaretskii
2020-11-24 19:05                       ` Trevor Bentley
2020-11-24 19:35                         ` Eli Zaretskii
2020-11-25 10:22                           ` Trevor Bentley
2020-11-25 17:47                             ` Eli Zaretskii
2020-11-25 19:06                               ` Trevor Bentley
2020-11-25 19:22                                 ` Eli Zaretskii
2020-11-25 19:38                                   ` Trevor Bentley
2020-11-25 20:02                                     ` Eli Zaretskii
2020-11-25 20:43                                       ` Trevor Bentley
2020-11-25 17:48                           ` Carlos O'Donell
2020-11-25 17:45                       ` Carlos O'Donell
2020-11-25 18:03                         ` Eli Zaretskii
2020-11-25 18:57                           ` Carlos O'Donell
2020-11-25 19:13                             ` Eli Zaretskii
2020-11-26  9:09                           ` Jean Louis
2020-11-26 14:13                             ` Eli Zaretskii
2020-11-26 18:37                               ` Jean Louis
2020-11-27  5:08                                 ` Carlos O'Donell
2020-11-25 18:08                         ` Jean Louis
2020-11-25 18:51                           ` Trevor Bentley
2020-11-25 19:02                             ` Carlos O'Donell
2020-11-25 19:17                               ` Trevor Bentley
2020-11-25 20:51                                 ` Carlos O'Donell
2020-11-26 13:58                                   ` Eli Zaretskii
2020-11-26 20:21                                     ` Carlos O'Donell
2020-11-26 20:30                                       ` Eli Zaretskii
2020-11-27  5:04                                         ` Carlos O'Donell
2020-11-27  7:40                                           ` Eli Zaretskii
2020-11-27  7:52                                             ` Eli Zaretskii
2020-11-27  8:20                                               ` Eli Zaretskii
2020-11-28  9:00                                                 ` Eli Zaretskii
2020-11-28 10:45                                                   ` Jean Louis
2020-11-28 17:49                                                   ` Trevor Bentley
2020-11-30 17:17                                                     ` Trevor Bentley
2020-11-30 18:15                                                       ` Eli Zaretskii
2020-11-30 18:33                                                         ` Trevor Bentley
2020-11-30 19:02                                                           ` Eli Zaretskii
2020-11-30 19:17                                                             ` Jean Louis
2020-12-01 10:14                                                               ` Trevor Bentley
2020-12-01 10:33                                                                 ` Jean Louis
2020-12-01 16:00                                                               ` Eli Zaretskii
2020-12-01 16:14                                                                 ` Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors
2020-12-08 21:50                                                       ` Trevor Bentley
2020-12-08 22:12                                                         ` Carlos O'Donell
2020-12-10 18:45                                                         ` Eli Zaretskii
2020-12-10 19:21                                                           ` Stefan Monnier
2020-12-10 19:33                                                             ` Trevor Bentley
2020-12-10 19:47                                                               ` Stefan Monnier
2020-12-10 20:26                                                             ` Jean Louis
2020-12-10 20:30                                                             ` Jean Louis
2020-12-12 11:20                                                             ` Trevor Bentley
2020-12-12 11:40                                                               ` Eli Zaretskii
2020-12-12 19:14                                                                 ` Stefan Monnier
2020-12-12 19:20                                                                   ` Eli Zaretskii
2020-12-12 19:46                                                                     ` Stefan Monnier
2020-12-12 19:51                                                                       ` Eli Zaretskii
2020-12-12 20:14                                                                         ` Trevor Bentley
2020-12-12 22:16                                                                 ` Michael Heerdegen
2020-12-13  3:34                                                                   ` Eli Zaretskii
2020-12-13 10:20                                                                     ` Trevor Bentley
2020-12-13 15:30                                                                       ` Eli Zaretskii
2020-12-13 19:34                                                                         ` Trevor Bentley
2020-12-13 19:38                                                                           ` Eli Zaretskii
2020-12-13 19:59                                                                             ` Trevor Bentley
2020-12-13 20:21                                                                               ` Eli Zaretskii
2020-12-13 20:41                                                                                 ` Trevor Bentley
2020-12-14  3:24                                                                                   ` Eli Zaretskii
2020-12-14 21:24                                                                                     ` Trevor Bentley
2021-01-20 12:02                                                                                       ` Trevor Bentley
2021-01-20 12:08                                                                                         ` Trevor Bentley
2021-01-20 14:53                                                                                         ` Stefan Monnier
2021-01-20 15:32                                                                                           ` Eli Zaretskii
2021-01-20 15:40                                                                                             ` Stefan Monnier
2020-09-12  2:12                                                                                               ` bug#43395: 28.0.50; memory leak Madhu
2020-09-14 15:08                                                                                                 ` Eli Zaretskii
2020-09-15  1:23                                                                                                   ` Madhu
     [not found]                                                                                                 ` <handler.43395.D43389.161115724232582.notifdone@debbugs.gnu.org>
2021-02-06 16:25                                                                                                   ` bug#43389: bug#43395: closed Madhu
2021-03-21 14:10                                                                                                     ` Madhu
2021-01-20 15:49                                                                                               ` bug#43389: 28.0.50; Emacs memory leaks using hard disk all time Trevor Bentley
2020-12-10 20:24                                                           ` Jean Louis
2020-12-12  1:28                                                           ` Jean Louis
2020-12-12  8:49                                                             ` Andreas Schwab
2020-12-03  6:30                                                   ` Jean Louis
2020-11-28 17:31                                               ` Trevor Bentley
2020-11-27 15:33                                           ` Eli Zaretskii
2020-12-08 22:15                                             ` Carlos O'Donell
2020-11-25 19:01                           ` Carlos O'Donell
2020-11-26 12:37                         ` Trevor Bentley [this message]
2020-11-26 14:30                           ` Eli Zaretskii
2020-11-26 15:19                             ` Trevor Bentley
2020-11-26 15:31                               ` Eli Zaretskii
2020-11-27  4:54                               ` Carlos O'Donell
2020-11-27  8:44                                 ` Jean Louis
2020-11-26 18:25                             ` Jean Louis
2020-11-27  4:55                               ` Carlos O'Donell
2020-11-23  3:35             ` Carlos O'Donell
2020-11-23 11:07               ` Jean Louis
2020-11-19  7:43       ` bug#44666: 28.0.50; malloc-info: Emacs became not responsive, " Jean Louis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/emacs/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87a6v4thst.fsf@mail.trevorbentley.com \
    --to=trevor@trevorbentley.com \
    --cc=43389@debbugs.gnu.org \
    --cc=bugs@gnu.support \
    --cc=carlos@redhat.com \
    --cc=dj@redhat.com \
    --cc=eliz@gnu.org \
    --cc=fweimer@redhat.com \
    --cc=michael_heerdegen@web.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).