From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#45705: [feature/native-comp] Excessive memory consumption on windows 10 Date: Sat, 09 Jan 2021 10:55:23 +0000 Message-ID: References: <87y2h5hjve.fsf@gmail.com> <831revjyja.fsf@gnu.org> <83turrif53.fsf@gnu.org> <83lfd2ilwf.fsf@gnu.org> Reply-To: Andrea Corallo Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="16830"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: edouard.debry@gmail.com, 45705@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Jan 09 11:56:09 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kyBui-0004JU-Qb for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 09 Jan 2021 11:56:08 +0100 Original-Received: from localhost ([::1]:58348 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kyBuh-0001W8-JN for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 09 Jan 2021 05:56:07 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34194) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kyBub-0001W2-Vr for bug-gnu-emacs@gnu.org; Sat, 09 Jan 2021 05:56:01 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:39715) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kyBub-00066I-Og for bug-gnu-emacs@gnu.org; Sat, 09 Jan 2021 05:56:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kyBub-0007hs-Mm for bug-gnu-emacs@gnu.org; Sat, 09 Jan 2021 05:56:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Andrea Corallo Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 09 Jan 2021 10:56:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45705 X-GNU-PR-Package: emacs Original-Received: via spool by 45705-submit@debbugs.gnu.org id=B45705.161018972729574 (code B ref 45705); Sat, 09 Jan 2021 10:56:01 +0000 Original-Received: (at 45705) by debbugs.gnu.org; 9 Jan 2021 10:55:27 +0000 Original-Received: from localhost ([127.0.0.1]:51261 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kyBu3-0007gw-1S for submit@debbugs.gnu.org; Sat, 09 Jan 2021 05:55:27 -0500 Original-Received: from mx.sdf.org ([205.166.94.24]:58233) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kyBu0-0007gm-Vt for 45705@debbugs.gnu.org; Sat, 09 Jan 2021 05:55:26 -0500 Original-Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTP id 109AtNS0024509; Sat, 9 Jan 2021 10:55:23 GMT In-Reply-To: <83lfd2ilwf.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 09 Jan 2021 09:56:16 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:197543 Archived-At: Eli Zaretskii writes: >> From: Andrea Corallo >> Cc: edouard.debry@gmail.com, 45705@debbugs.gnu.org >> Date: Fri, 08 Jan 2021 22:02:38 +0000 >> >> I've compiled current native-comp with and without --with-nativecomp >> repeating the experiment with and without X. These are the data-points: >> >> | | --without-x | --without-x --with-nativecomp | | >> |---------+-------------+-------------------------------+------| >> | -Q | 49M | 92M | 1.9x | >> | my-conf | 92M | 179M | 1.9x | >> >> >> | | | --with-nativecomp | | >> |---------+------+-------------------+------| >> | -Q | 536M | 756M | 1.4x | >> | my-conf | 585M | 1453M | 2.4x | >> >> So yes shared are using considerably more memory, I think this is >> expected as also the file footprint suggests native code is less dense >> that byte-code (actually with a quite similar relative results). > > Thanks for the data points. > > What about memory usage when there's a background compilation of Lisp > going on? GCC is known to be a memory hog in some cases, so I wonder > what happens in this case with libgccjit. In June we changed the way we store immediate objects in the shared and this makes the compilation way lighter on the GCC side (both in time and memory). I've no precise data on this other than the experimental observation that compiling all Elisp files in Emacs on 32bit systems is not anymore an issue. This IIUC implies that the memory footprint for each compilation is always < 2GB. As a note: in all cases except bootstrap the final pass (the one driving libgccjit) is executed as a sub-process, this to protect us from eventual GCC leaks and not to generate unnecessary fragmentation. In async compilation we indeed run all the compilation (also the Lisp computation) in the child process, so compiling should not have impact on the memory footprint of the main Emacs session. > (Do we allow multiple async compilations, btw? if so, how many > concurrent compilations can be running, and how do we or the user > control that?) Yes see > Also, what are the numbers for a session that has been running for > several days? I understand that it would be hard for you to collect > such numbers about all the configurations, but could you show the > growth of the configuration you are routinely using, which I presume > is --with-x --with-nativecomp and with your config? As your numbers > above show, it starts at 1.5 GiB, but what is the footprint after a > day or a week? ATM I can provide this number, this is an Aarch64 daemon compiled with '--without-x' with an up-time of 25 days and is showing a footprint of 765M. >> Indeed *with use the delta should decay as data are the same and there's >> no difference in its representation*, so this picture should be more on >> the worst case side than on the average. > > That's why I asked to see the memory footprint after the session has > run for some time, yes. The hard part is to have a reference to compare against as the memory footprint is strictly connected to the usage. One with very regular working habits should work like one week on vanilla and one week on native-comp to make a comparison. I've no regular working habits so I fear I'm not the best fit for this comparison. Thanks Andrea