From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.devel Subject: Re: Memory usage report Date: Sat, 19 Sep 2020 22:34:28 +0800 Message-ID: <87mu1lesy3.fsf@localhost> References: <87lfh8kyot.fsf@gnus.org> <83k0ws5hzt.fsf@gnu.org> <87h7rwkxtk.fsf@gnus.org> <83imcb61p7.fsf@gnu.org> <871rizl5mf.fsf@gnus.org> <83r1qz48h3.fsf@gnu.org> <871rizjogr.fsf@gnus.org> <83pn6j45rr.fsf@gnu.org> <87wo0ri6kz.fsf@gnus.org> <83o8m34433.fsf@gnu.org> <87imcb43e3.fsf@localhost> <83k0wr4222.fsf@gnu.org> <87ft7f40p8.fsf@localhost> <83h7rv3xsc.fsf@gnu.org> <87wo0rdpud.fsf@localhost> <838sd6522b.fsf@gnu.org> <87tuvuehhq.fsf@localhost> <83wo0q2ohm.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="19121"; mail-complaints-to="usenet@ciao.gmane.io" Cc: larsi@gnus.org, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Sep 19 16:36:29 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kJdyX-0004sH-86 for ged-emacs-devel@m.gmane-mx.org; Sat, 19 Sep 2020 16:36:29 +0200 Original-Received: from localhost ([::1]:41780 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJdyW-0001Ed-5M for ged-emacs-devel@m.gmane-mx.org; Sat, 19 Sep 2020 10:36:28 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34740) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJdxh-0000Ut-Ec for emacs-devel@gnu.org; Sat, 19 Sep 2020 10:35:37 -0400 Original-Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]:46700) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kJdxf-0003ma-LY; Sat, 19 Sep 2020 10:35:37 -0400 Original-Received: by mail-pl1-x62a.google.com with SMTP id f1so4509793plo.13; Sat, 19 Sep 2020 07:35:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=D+9f3wmrMEH5ZXD1prWn3FbYZdSOfAirNBGGMki67ns=; b=jj0Qb9PinpRMlBr2qgUGRuA/7KEZ2u0SAF+Ws5xBU61aUT6Q20Rv1QsX0wb4jYyfGU 1J3ZnkV9psUjK4umGVlyJRTLCGXha8YnjEEnIm600erMvaVXc7kvqLnyFjmek85Op7Ap IpenY2YokFX6umifN+elJGbZdEsT2EqDnat89FSLJi4g2ot9MNtt92Cpp0qwOFC4e5h8 xFCEEDVu1kYXQ/ov467SN92qeJ7q80J69sSd2hYaT6e5fqZ66XlYe+BRnKbN/r8eE5WR pX5//9ZPB7Gfv726k8FKJum6SJl3acvaAgiGrM4+xjQMpUddEb/mbW/F6uzJSQuqv5mu 3dKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=D+9f3wmrMEH5ZXD1prWn3FbYZdSOfAirNBGGMki67ns=; b=V70Etm1qv96Ea9X3FsMsaEDiJ7B8oxDJ+zypWA838rpSj3stgkI+Givbr4wMWIAtn1 /G6TxcjRlFWVxonRQOLb440fxDe4bUyQYTyhseENv/LnURHu1OSwOoTEV0ThxvLoV7AR dURCMoyBWKcr1GfxWub2gnqv871ywa+C5ESU1pnHPhaV1sejj0+O9c0vLFjq3bAbI4Vv lhww9ga1GfKpqbreBTkBbpMRe1GHl1ZJOSBwyqut3MpxDnjr/51wZCd4wWs+TMXWDTXq tbpauusKX6SWUwdW40Q+90VOc0bJ75fWKPq5PMr56ka2Q9mvuRgSA74eEkKe4Smm7G6H hfIw== X-Gm-Message-State: AOAM530E2RqweGhW1DkVOPau9k7sEGKfSd6J8RxsdCvHzxMzEoR4pVnG SLKXG4dtbz7+Msf1tnRrLkeXkBHtFDU= X-Google-Smtp-Source: ABdhPJyr55HsY3yrwWmV2w0mYtsHPw6HFzG593I3mJFfyNlR6d4IODR1jMr/Qogqgs3/TIZuS/sA4g== X-Received: by 2002:a17:902:9686:b029:d1:e598:4002 with SMTP id n6-20020a1709029686b02900d1e5984002mr20384746plp.60.1600526132894; Sat, 19 Sep 2020 07:35:32 -0700 (PDT) Original-Received: from localhost ([208.167.241.222]) by smtp.gmail.com with ESMTPSA id e13sm5766965pjy.38.2020.09.19.07.35.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Sep 2020 07:35:31 -0700 (PDT) In-Reply-To: <83wo0q2ohm.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=yantar92@gmail.com; helo=mail-pl1-x62a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:256173 Archived-At: >> top shows more than 200Mb memory increase. > > Which are explained by the statistics produced by GC. IOW, you have > many more live Lisp objects, which take up those megabytes. I meant that 200Mb is Lisp objects, but top shows more then 200Mb (around 300Mb). So, there is extra 100Mb coming from somewhere else. > Since that's related to Org buffers, the best place to discuss this is > on Org mailing lists. Perhaps there are ways to make Org use less > memory, but the expertise for that is there. The problem is how to identify where the memory usage is coming from. Indeed, org is using overlays and text properties extensively. But how much do those influence the memory usage? I am somewhat familiar with org-mode codebase, but I have no idea how to debug memory usage issues on practice. Total memory usage report provides little clue about what should be changed. > The "memory" profiler doesn't measure the usage of memory, it measures > CPU usage triggered by memory allocation calls (instead of the > periodic profiling signal). So this profile is not supposed to be > useful for profiling memory usage. Thanks! I did not know that. Would it be possible to have a "real" memory profiler showing how much the memory usage changed after-before running separate functions? This could help debugging memory usage (i.e. in org-mode). Best, Ihor Eli Zaretskii writes: >> From: Ihor Radchenko >> Cc: larsi@gnus.org, emacs-devel@gnu.org >> Date: Sat, 19 Sep 2020 08:29:37 +0800 >> >> > Where do you see evidence for a significant memory in >> > the heap? >> >> top shows more than 200Mb memory increase. > > Which are explained by the statistics produced by GC. IOW, you have > many more live Lisp objects, which take up those megabytes. > >> While I would also like to answer this question, I also have large >> fraction of lisp objects. As I mentioned in previous emails (sorry if I >> was not clear), I would appreciate some way to understand why my **lisp >> object** memory grew from 283Mb right after loading my org files to 1Gb >> later. > > Since that's related to Org buffers, the best place to discuss this is > on Org mailing lists. Perhaps there are ways to make Org use less > memory, but the expertise for that is there. > > Producing memory usage for buffer-local variables will AFAIU need > support on the C level, so if someone wants to work on that, please > do. However, issues like this one are very infrequent in Emacs > maintenance; most of the memory-related problems are not due to some > package using up many Lisp objects. > >> I tried to use memory profiler to understand the memory usage, but the >> memory profiler report appear to show peak memory usage (there tend to >> be functions with many let-bindings). So, it seems not be to useful for >> me (correct me if I am wrong). > > The "memory" profiler doesn't measure the usage of memory, it measures > CPU usage triggered by memory allocation calls (instead of the > periodic profiling signal). So this profile is not supposed to be > useful for profiling memory usage.