From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.devel Subject: Re: Memory usage report Date: Sat, 19 Sep 2020 23:14:52 +0800 Message-ID: <87k0wper2r.fsf@localhost> References: <87lfh8kyot.fsf@gnus.org> <83k0ws5hzt.fsf@gnu.org> <87h7rwkxtk.fsf@gnus.org> <83imcb61p7.fsf@gnu.org> <871rizl5mf.fsf@gnus.org> <83r1qz48h3.fsf@gnu.org> <871rizjogr.fsf@gnus.org> <83pn6j45rr.fsf@gnu.org> <87wo0ri6kz.fsf@gnus.org> <83o8m34433.fsf@gnu.org> <87imcb43e3.fsf@localhost> <83k0wr4222.fsf@gnu.org> <87ft7f40p8.fsf@localhost> <83h7rv3xsc.fsf@gnu.org> <87wo0rdpud.fsf@localhost> <838sd6522b.fsf@gnu.org> <87tuvuehhq.fsf@localhost> <83wo0q2ohm.fsf@gnu.org> <87mu1lesy3.fsf@localhost> <83d02h3jj2.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35684"; mail-complaints-to="usenet@ciao.gmane.io" Cc: larsi@gnus.org, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Sep 19 17:19:09 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kJedp-0009AZ-6O for ged-emacs-devel@m.gmane-mx.org; Sat, 19 Sep 2020 17:19:09 +0200 Original-Received: from localhost ([::1]:50928 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJedo-0002eL-7a for ged-emacs-devel@m.gmane-mx.org; Sat, 19 Sep 2020 11:19:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:42358) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJeao-0008Tw-Op for emacs-devel@gnu.org; Sat, 19 Sep 2020 11:16:02 -0400 Original-Received: from mail-qk1-x734.google.com ([2607:f8b0:4864:20::734]:35648) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kJeak-0001Dv-C3; Sat, 19 Sep 2020 11:16:02 -0400 Original-Received: by mail-qk1-x734.google.com with SMTP id q5so10084363qkc.2; Sat, 19 Sep 2020 08:15:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=L238SvpQxl4W0XN7J/gnKCXXvJMT6JzrNVo+lgRZaEU=; b=IYFO6lh1Lvjczvfjm7WYgJ9h1tXGD57kmzFkoabFhw0P0emvWNaUxF1nhm0pkk1tCh przA353DlC9MrPHI1PF7iDUId6C3tUDgJoTbRfQnbw2me1o1bxTjwiIAbtOW4T0fpHk8 5AvDXri5ldOXaJDTSNO9E86nGueiS32v2a0TMnWB2NLkT1zX46CbUIIei5TQnaEj5tj9 sdzC4TNVq5+AMXf7CVlxu7bFtzI9FlrZed4POD4M9bTt2bd+YRbu3O/Nrvpyn1yBmid7 0KlkbAIoYWiF/4kY/AaBEIf4r7qAiJb9k/pLNFPvjsqaJI2lNgLF7r26V5E1H+Q/b/qK n71A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=L238SvpQxl4W0XN7J/gnKCXXvJMT6JzrNVo+lgRZaEU=; b=Idf1PT+itcbmgckYwtrfeTIL+SRXEgBLlDhhoXh0VqIalcuBFEDop/u3ZRURefz5U8 ZbScAPL9+7NDezx2gmrjao3M4Jy0yzRvuQg3JHbVsycQNIzcnNdgm7ChXpo8AdZBFngW Kt2d5OVMUJY65UwM88uOJpdl3Zg2xGC2gH94dRrDte1HmIKvJ3L++ZWvBeduYxG+jrEw uDvXfQNxiiF+8LK7VgzOysrkKYynlKJUUUHbymegunUTD6D06T3z4dZOjtkp+/8M1gKE LouVzEtiVMz3M57BSjV4QdidPYPDJMxOj7rYOvG/loaggEnIAgWLFWrmqQosqWWm/P5G deMA== X-Gm-Message-State: AOAM533XzBoEDpL6CdtfZrGDaTw/56rZoaD3ItrbTU43tSw1s3yXRqAm LX2fBWcxNQIT9Vn6VmYOwENznTt2J6g= X-Google-Smtp-Source: ABdhPJx496EAoaFo7k3i0+q01P3lJTQjMCYzlGDaXziA4LA5nfG4u8pzcYlAQqB7M/fPEqUCO6O4dQ== X-Received: by 2002:a37:a347:: with SMTP id m68mr38364930qke.81.1600528556225; Sat, 19 Sep 2020 08:15:56 -0700 (PDT) Original-Received: from localhost ([208.167.241.222]) by smtp.gmail.com with ESMTPSA id 71sm4324120qki.85.2020.09.19.08.15.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Sep 2020 08:15:55 -0700 (PDT) In-Reply-To: <83d02h3jj2.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::734; envelope-from=yantar92@gmail.com; helo=mail-qk1-x734.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:256181 Archived-At: > I think the more useful question is: are all those overlays and text > properties necessary? If they are, they take the memory they are > supposed to take. Some of the overlays and text properties are not strictly necessary. For example, text properties are sometimes used as cache to avoid parsing text multiple times. Is the resulting speedup worth extra memory usage? It is not clear since we do not have an easy way to determine the extra memory usage. >> Thanks! I did not know that. Would it be possible to have a "real" >> memory profiler showing how much the memory usage changed after-before >> running separate functions? > If you mean memory used by Lisp objects, then GC reports that. > > If you mean memory used by non-Lisp objects, I don't see how we could > produce that without having infrastructure for tracking memory > allocation, something that debugging malloc libraries already do. Hmm. I am looking again at the profiler-report output. It seems to report the memory allocation for individual function calls (that's what I meant by "real" memory profiler). Do I miss something? Function Bytes % - command-execute 516,227,475 77% - call-interactively 516,222,195 77% - funcall-interactively 516,222,195 77% - helm-M-x 203,107,508 30% - helm-M-x-read-extended-command 203,107,508 30% - helm 202,313,652 30% - apply 202,310,580 30% - helm 202,310,580 30% - apply 202,310,580 30% - helm-internal 202,310,580 30% - helm-display-buffer 112,541,910 16% - helm-default-display-buffer 112,540,854 16% - display-buffer 112,540,854 16% - display-buffer--maybe-pop-up-frame-or-window 112,540,854 16% - display-buffer--maybe-pop-up-window 112,540,854 16% - display-buffer-pop-up-window 112,540,854 16% + window--try-to-split-window 56,401,971 8% window--display-buffer 56,130,435 8% window--maybe-raise-frame 5,280 0% Best, Ihor Eli Zaretskii writes: >> From: Ihor Radchenko >> Cc: larsi@gnus.org, emacs-devel@gnu.org >> Date: Sat, 19 Sep 2020 22:34:28 +0800 >> >> >> top shows more than 200Mb memory increase. >> > >> > Which are explained by the statistics produced by GC. IOW, you have >> > many more live Lisp objects, which take up those megabytes. >> >> I meant that 200Mb is Lisp objects, but top shows more then 200Mb >> (around 300Mb). So, there is extra 100Mb coming from somewhere else. > > I got a different (smaller) value, but it isn't worth to continue this > argument. > >> > Since that's related to Org buffers, the best place to discuss this is >> > on Org mailing lists. Perhaps there are ways to make Org use less >> > memory, but the expertise for that is there. >> >> The problem is how to identify where the memory usage is coming from. >> Indeed, org is using overlays and text properties extensively. But how >> much do those influence the memory usage? > > I think the more useful question is: are all those overlays and text > properties necessary? If they are, they take the memory they are > supposed to take. > >> > The "memory" profiler doesn't measure the usage of memory, it measures >> > CPU usage triggered by memory allocation calls (instead of the >> > periodic profiling signal). So this profile is not supposed to be >> > useful for profiling memory usage. >> >> Thanks! I did not know that. Would it be possible to have a "real" >> memory profiler showing how much the memory usage changed after-before >> running separate functions? > > If you mean memory used by Lisp objects, then GC reports that. > > If you mean memory used by non-Lisp objects, I don't see how we could > produce that without having infrastructure for tracking memory > allocation, something that debugging malloc libraries already do.