Re: Memory usage report

[Ihor Radchenko <yantar92@gmail.com>]

> Did you actually see cases where something like that happens?  I mean
> that loading some library, or all of your Org files, causes increase
> in memory usage of hundreds of MBs (320MB in your example)?  I only
> see this when the buffers themselves are near that value.

Yep. I see it every time I run Emacs. I have done M-x memory-usage
before and after loading org-agenda (which loads all my working org
files).

Before:
Total in lisp objects: 77.4MB (live 47.7MB, dead 29.7MB)
Buffer ralloc memory usage:
16 buffers
39.3kB total (23.6kB in gaps)

After:
Total in lisp objects:  283MB (live  218MB, dead 64.7MB)
Buffer ralloc memory usage:
47 buffers
6.79MB total (78.6kB in gaps)

This is not all the memory increase. Some part seems to be in heap.

Also, the memory usage is even worse when org-mode is using overlays (I
am on org-mode branch using text properties instead of overlays in org).

Full memory-usage reports are shown below.

The memory consumption gets even worse after I use Emacs for several
hours. Recently, it tend to settle around 20% (of 8Gb) with lisp objects
taking around 1Gb.

This last part is the most annoying and also difficult to track - I have
no easy way to know if it is also caused by org-mode or it is something
else.

Best,
Ihor

Before:

Garbage collection stats:
((conses 16 934007 1096370) (symbols 48 73755 2041) (strings 32 265462 116861) (string-bytes 1 9693220) (vectors 16 115429) (vector-slots 8 1423224 1215629) (floats 8 1419 3034) (intervals 56 1366 249) (buffers 992 16))

 =>	14.3MB (+ 16.7MB dead) in conses
	3.38MB (+ 95.7kB dead) in symbols
	8.10MB (+ 3.57MB dead) in strings
	9.24MB in string-bytes
	1.76MB in vectors
	10.9MB (+ 9.27MB dead) in vector-slots
	11.1kB (+ 23.7kB dead) in floats
	74.7kB (+ 13.6kB dead) in intervals
	15.5kB in buffers

Total in lisp objects: 77.4MB (live 47.7MB, dead 29.7MB)

Buffer ralloc memory usage:
16 buffers
39.3kB total (23.6kB in gaps)
      Size	Gap	Name

     13213	2000	*scratch*
      1252	769	*Messages*
       676	1574	*Buffer Details*
       294	1940	 *straight-watcher*
       265	1756	*straight-process*
       124	1911	 *code-conversion-work*
       112	3941	*helm M-x*
        82	2000	*elfeed-log*
        67	2000	 *pdf-info-query--parse-response*
        53	2000	 *Echo Area 1*
        24	2001	 *Echo Area 0*
         8	12	 *pdf-info-query--escape*
         0	20	 *Minibuf-1*
         0	20	 *Minibuf-0*
         0	20	 *epdfinfo*
         0	2067	 tq-temp-epdfinfo

After:

Garbage collection stats:
((conses 16 9393579 2919763) (symbols 48 75965 293) (strings 32 323708 224589) (string-bytes 1 13017637) (vectors 16 154787) (vector-slots 8 2249055 1365412) (floats 8 1630 2854) (intervals 56 547977 53455) (buffers 992 47))

 =>	 143MB (+ 44.6MB dead) in conses
	3.48MB (+ 13.7kB dead) in symbols
	9.88MB (+ 6.85MB dead) in strings
	12.4MB in string-bytes
	2.36MB in vectors
	17.2MB (+ 10.4MB dead) in vector-slots
	12.7kB (+ 22.3kB dead) in floats
	29.3MB (+ 2.85MB dead) in intervals
	45.5kB in buffers

Total in lisp objects:  283MB (live  218MB, dead 64.7MB)

Buffer ralloc memory usage:
47 buffers
6.79MB total (78.6kB in gaps)
      Size	Gap	Name

   2345866	2000	nodeadline.org
   1914956	2000	notes.org
    695164	2000	TODO.org
    548214	2000	config.org
    487501	2000	get_started_org_mode.org
    361666	2000	CuNb-beamcrack.org
    340586	2000	References.org
     84150	2000	articles.org
     47507	2000	Journal.org
     45875	2000	Олди Генри Лайон.org
     31561	2000	mirror-text.org
     30235	2000	rss.org
     29662	2000	org-autosort.org
     18633	2000	[book]DFR.org
     14412	801	*scratch*
      9932	2000	schedule.org
      8719	2000	Abraham Daniel.org
      7292	2000	org-mode.org
      6172	2000	inbox.org
      4503	2000	contacts.org
      2564	2000	 *code-conversion-work*
      1944	2000	gnuplot.org
      1838	183	*Messages*
      1125	950	*Memory-Profiler-Report 2020-09-18 23:59:40*
       897	1131	 *emacsql-log*
       889	1727	 *Agenda Commands*
       681	1569	*Buffer Details*
       301	1720	*Org Agenda(d:)*
       294	1940	 *straight-watcher*
       265	1756	*straight-process*
       125	3928	*helm M-x*
       119	2000	frommobile.org
        82	2000	*elfeed-log*
        67	2000	 *pdf-info-query--parse-response*
        50	1973	*Helm Completions*
        41	1988	*helm-mode-profiler-report-write-profile*
        35	1990	 *Echo Area 0*
        31	1991	*helm-mode-profiler-start*
        13	2474	 *emacsql-sqlite*
         8	12	 *pdf-info-query--escape*
         6	14	 *temp*
         0	2030	 *Minibuf-1*
         0	20	 *Minibuf-0*
         0	2053	 *Echo Area 1*
         0	20	 *epdfinfo*
         0	2067	 tq-temp-epdfinfo
         0	20	 *server*



Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@gmail.com>
>> Cc: larsi@gnus.org, emacs-devel@gnu.org
>> Date: Fri, 18 Sep 2020 22:30:11 +0800
>> 
>> Seeing how much memory is used by buffer-local variables in org
>> buffers would help. Similarly, total memory consumption in variables
>> from individual packages would be helpful to pinpoint why the memory
>> usage is high.
>
> That'd require some filtering of the variables, not just finding the
> largest one.
>
> Did you actually see cases where something like that happens?  I mean
> that loading some library, or all of your Org files, causes increase
> in memory usage of hundreds of MBs (320MB in your example)?  I only
> see this when the buffers themselves are near that value.