all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Trevor Bentley <trevor@trevorbentley.com>
To: Carlos O'Donell <carlos@redhat.com>, Eli Zaretskii <eliz@gnu.org>
Cc: fweimer@redhat.com, 43389@debbugs.gnu.org, bugs@gnu.support,
	dj@redhat.com, michael_heerdegen@web.de,
Subject: bug#43389: 28.0.50; Emacs memory leaks using hard disk all time
Date: Thu, 26 Nov 2020 13:37:54 +0100	[thread overview]
Message-ID: <87a6v4thst.fsf@mail.trevorbentley.com> (raw)
In-Reply-To: <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com>

> You want visibility into what is USING that memory. 
> 
> With glibc-malloc-trace-utils you can try to do that with: 
> 
> LD_PRELOAD=libmtrace.so \ MTRACE_CTL_FILE=/home/user/app.mtr \ 
> MTRACE_CTL_BACKTRACE=1 \ ./app 
> 
> This will use libgcc's unwinder to get a copy of the malloc 
> caller address and then we'll have to decode that based on a 
> /proc/self/maps. 
> 
> Next steps: - Get a glibc-malloc-trace-utils trace of the 
> application ratcheting.  - Get a copy of /proc/$PID/maps for the 
> application (shorter version of smaps). 
> 

Oh, this is going to be a problem.  I guess it is producing one 
trace file per thread?

I ran it with libmtrace overnight.  Memory usage was very high, 
but it doesn't look like the same problem.  I hit 1550MB of RSS, 
but smaps reported only ~350MB of that was in the heap, which 
seemed reasonable for the ~150MB that emacs reported it was using. 
Does libmtrace add a lot of memory overhead?

However, libmtrace has made 4968 files totalling 26GB in that 
time.  Ouch.

It's going to be hard to tell when I hit the bug under libmtrace, 
questionable whether the report will even fit on my disk, and 
tricky to share however many tens of gigabytes of trace files it 
results in.

If it's one trace per thread, though, then we at least know that 
my emacs process in question is blazing through threads.  That 
could be relevant.

Other thing to note (for Eli): I wrapped garbage-collect like so:

---
(defun trev/garbage-collect (orig-fun &rest args) 
  (message "%s -- Starting garbage-collect." 
  (current-time-string)) (let ((time (current-time)) 
        (result (apply orig-fun args))) 
    (message "%s -- Finished garbage-collect in %.06f" 
    (current-time-string) (float-time (time-since time))) result)) 
(add-function :around (symbol-function 'garbage-collect) 
#'trev/garbage-collect)
---

This printed a start and stop message each time I evaluated 
garbage-collect manually.  It did not print any messages in 11 
hours of running unattended.  This is with an active network 
connection receiving messages fairly frequently, so there was 
plenty of consing going on.  Hard for me to judge if it should run 
any garbage collection in that time, but I would have expected so.

-Trevor





  parent reply	other threads:[~2020-11-26 12:37 UTC|newest]

Thread overview: 166+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-15 14:55 bug#44666: 28.0.50; malloc-info: Emacs became not responsive, using hard disk all time Jean Louis
2020-11-16 16:11 ` Eli Zaretskii
2020-11-16 16:17   ` Jean Louis
2020-11-17 15:04     ` Eli Zaretskii
2020-11-19  6:59       ` Jean Louis
2020-11-19 14:37         ` bug#43389: 28.0.50; Emacs memory leaks " Eli Zaretskii
2020-11-20  3:16           ` Jean Louis
2020-11-20  8:10             ` Eli Zaretskii
2020-11-22 19:52               ` Jean Louis
2020-11-22 20:16                 ` Eli Zaretskii
2020-11-23  3:41                   ` Carlos O'Donell
2020-11-23  8:11                   ` Jean Louis
2020-11-23  9:59                     ` Eli Zaretskii
2020-11-23 17:19                       ` Arthur Miller
2020-11-23 17:44                         ` Eli Zaretskii
2020-11-23 18:34                           ` Arthur Miller
2020-11-23 19:06                             ` Jean Louis
2020-11-23 19:15                             ` Eli Zaretskii
2020-11-23 19:49                               ` Arthur Miller
2020-11-23 20:04                                 ` Eli Zaretskii
2020-11-23 21:12                                   ` Arthur Miller
2020-11-24  2:07                                   ` Arthur Miller
2020-11-23 20:31                                 ` Jean Louis
2020-11-23 21:22                                   ` Arthur Miller
2020-11-24  5:29                                     ` Jean Louis
2020-11-24  8:15                                       ` Arthur Miller
2020-11-24  9:06                                         ` Jean Louis
2020-11-24  9:27                                           ` Arthur Miller
2020-11-24 17:18                                             ` Jean Louis
2020-11-25 14:59                                               ` Arthur Miller
2020-11-25 15:09                                                 ` Jean Louis
2020-11-23 13:27                   ` Jean Louis
2020-11-23 15:54                     ` Carlos O'Donell
2020-11-23 18:58                       ` Jean Louis
2020-11-23 19:34                         ` Eli Zaretskii
2020-11-23 19:49                           ` Jean Louis
2020-11-23 20:04                           ` Carlos O'Donell
2020-11-23 20:16                             ` Eli Zaretskii
2020-11-23 19:37                         ` Carlos O'Donell
2020-11-23 19:55                           ` Jean Louis
2020-11-23 20:06                             ` Carlos O'Donell
2020-11-23 20:18                               ` Jean Louis
2020-11-23 20:31                                 ` Eli Zaretskii
2020-11-23 20:41                                   ` Jean Louis
2020-11-23 20:53                                     ` Andreas Schwab
2020-11-23 21:09                                       ` Jean Louis
2020-11-24  3:25                                       ` Eli Zaretskii
2020-11-23 20:10                             ` Eli Zaretskii
2020-11-23 19:50                     ` Carlos O'Donell
2020-11-23 19:59                       ` Jean Louis
2020-11-23 10:59               ` Jean Louis
2020-11-23 15:46                 ` Eli Zaretskii
2020-11-23 17:29                   ` Arthur Miller
2020-11-23 17:45                     ` Eli Zaretskii
2020-11-23 18:40                       ` Arthur Miller
2020-11-23 19:23                         ` Eli Zaretskii
2020-11-23 19:38                           ` Arthur Miller
2020-11-23 19:52                             ` Eli Zaretskii
2020-11-23 20:03                               ` Arthur Miller
2020-11-23 19:39                           ` Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors
2020-11-23 19:59                             ` Arthur Miller
2020-11-23 20:15                               ` Eli Zaretskii
2020-11-23 21:15                                 ` Arthur Miller
2020-11-23 20:53                               ` Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors
2020-11-23 18:33                   ` Jean Louis
2020-11-23 21:30                   ` Trevor Bentley
2020-11-23 22:11                     ` Trevor Bentley
2020-11-24 16:07                     ` Eli Zaretskii
2020-11-24 19:05                       ` Trevor Bentley
2020-11-24 19:35                         ` Eli Zaretskii
2020-11-25 10:22                           ` Trevor Bentley
2020-11-25 17:47                             ` Eli Zaretskii
2020-11-25 19:06                               ` Trevor Bentley
2020-11-25 19:22                                 ` Eli Zaretskii
2020-11-25 19:38                                   ` Trevor Bentley
2020-11-25 20:02                                     ` Eli Zaretskii
2020-11-25 20:43                                       ` Trevor Bentley
2020-11-25 17:48                           ` Carlos O'Donell
2020-11-25 17:45                       ` Carlos O'Donell
2020-11-25 18:03                         ` Eli Zaretskii
2020-11-25 18:57                           ` Carlos O'Donell
2020-11-25 19:13                             ` Eli Zaretskii
2020-11-26  9:09                           ` Jean Louis
2020-11-26 14:13                             ` Eli Zaretskii
2020-11-26 18:37                               ` Jean Louis
2020-11-27  5:08                                 ` Carlos O'Donell
2020-11-25 18:08                         ` Jean Louis
2020-11-25 18:51                           ` Trevor Bentley
2020-11-25 19:02                             ` Carlos O'Donell
2020-11-25 19:17                               ` Trevor Bentley
2020-11-25 20:51                                 ` Carlos O'Donell
2020-11-26 13:58                                   ` Eli Zaretskii
2020-11-26 20:21                                     ` Carlos O'Donell
2020-11-26 20:30                                       ` Eli Zaretskii
2020-11-27  5:04                                         ` Carlos O'Donell
2020-11-27  7:40                                           ` Eli Zaretskii
2020-11-27  7:52                                             ` Eli Zaretskii
2020-11-27  8:20                                               ` Eli Zaretskii
2020-11-28  9:00                                                 ` Eli Zaretskii
2020-11-28 10:45                                                   ` Jean Louis
2020-11-28 17:49                                                   ` Trevor Bentley
2020-11-30 17:17                                                     ` Trevor Bentley
2020-11-30 18:15                                                       ` Eli Zaretskii
2020-11-30 18:33                                                         ` Trevor Bentley
2020-11-30 19:02                                                           ` Eli Zaretskii
2020-11-30 19:17                                                             ` Jean Louis
2020-12-01 10:14                                                               ` Trevor Bentley
2020-12-01 10:33                                                                 ` Jean Louis
2020-12-01 16:00                                                               ` Eli Zaretskii
2020-12-01 16:14                                                                 ` Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors
2020-12-08 21:50                                                       ` Trevor Bentley
2020-12-08 22:12                                                         ` Carlos O'Donell
2020-12-10 18:45                                                         ` Eli Zaretskii
2020-12-10 19:21                                                           ` Stefan Monnier
2020-12-10 19:33                                                             ` Trevor Bentley
2020-12-10 19:47                                                               ` Stefan Monnier
2020-12-10 20:26                                                             ` Jean Louis
2020-12-10 20:30                                                             ` Jean Louis
2020-12-12 11:20                                                             ` Trevor Bentley
2020-12-12 11:40                                                               ` Eli Zaretskii
2020-12-12 19:14                                                                 ` Stefan Monnier
2020-12-12 19:20                                                                   ` Eli Zaretskii
2020-12-12 19:46                                                                     ` Stefan Monnier
2020-12-12 19:51                                                                       ` Eli Zaretskii
2020-12-12 20:14                                                                         ` Trevor Bentley
2020-12-12 22:16                                                                 ` Michael Heerdegen
2020-12-13  3:34                                                                   ` Eli Zaretskii
2020-12-13 10:20                                                                     ` Trevor Bentley
2020-12-13 15:30                                                                       ` Eli Zaretskii
2020-12-13 19:34                                                                         ` Trevor Bentley
2020-12-13 19:38                                                                           ` Eli Zaretskii
2020-12-13 19:59                                                                             ` Trevor Bentley
2020-12-13 20:21                                                                               ` Eli Zaretskii
2020-12-13 20:41                                                                                 ` Trevor Bentley
2020-12-14  3:24                                                                                   ` Eli Zaretskii
2020-12-14 21:24                                                                                     ` Trevor Bentley
2021-01-20 12:02                                                                                       ` Trevor Bentley
2021-01-20 12:08                                                                                         ` Trevor Bentley
2021-01-20 14:53                                                                                         ` Stefan Monnier
2021-01-20 15:32                                                                                           ` Eli Zaretskii
2021-01-20 15:40                                                                                             ` Stefan Monnier
2020-09-12  2:12                                                                                               ` bug#43395: 28.0.50; memory leak Madhu
2020-09-14 15:08                                                                                                 ` Eli Zaretskii
2020-09-15  1:23                                                                                                   ` Madhu
     [not found]                                                                                                 ` <handler.43395.D43389.161115724232582.notifdone@debbugs.gnu.org>
2021-02-06 16:25                                                                                                   ` bug#43389: bug#43395: closed Madhu
2021-03-21 14:10                                                                                                     ` Madhu
2021-01-20 15:49                                                                                               ` bug#43389: 28.0.50; Emacs memory leaks using hard disk all time Trevor Bentley
2020-12-10 20:24                                                           ` Jean Louis
2020-12-12  1:28                                                           ` Jean Louis
2020-12-12  8:49                                                             ` Andreas Schwab
2020-12-03  6:30                                                   ` Jean Louis
2020-11-28 17:31                                               ` Trevor Bentley
2020-11-27 15:33                                           ` Eli Zaretskii
2020-12-08 22:15                                             ` Carlos O'Donell
2020-11-25 19:01                           ` Carlos O'Donell
2020-11-26 12:37                         ` Trevor Bentley [this message]
2020-11-26 14:30                           ` Eli Zaretskii
2020-11-26 15:19                             ` Trevor Bentley
2020-11-26 15:31                               ` Eli Zaretskii
2020-11-27  4:54                               ` Carlos O'Donell
2020-11-27  8:44                                 ` Jean Louis
2020-11-26 18:25                             ` Jean Louis
2020-11-27  4:55                               ` Carlos O'Donell
2020-11-23  3:35             ` Carlos O'Donell
2020-11-23 11:07               ` Jean Louis
2020-11-19  7:43       ` bug#44666: 28.0.50; malloc-info: Emacs became not responsive, " Jean Louis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87a6v4thst.fsf@mail.trevorbentley.com \
    --to=trevor@trevorbentley.com \
    --cc=43389@debbugs.gnu.org \
    --cc=bugs@gnu.support \
    --cc=carlos@redhat.com \
    --cc=dj@redhat.com \
    --cc=eliz@gnu.org \
    --cc=fweimer@redhat.com \
    --cc=michael_heerdegen@web.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.