* bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation*
@ 2019-12-15 19:58 Peter Ludemann
2019-12-16 4:16 ` bug#38629: Loading/killing the compilation output doesn't lose memory Peter Ludemann
2019-12-16 14:24 ` bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Noam Postavsky
0 siblings, 2 replies; 7+ messages in thread
From: Peter Ludemann @ 2019-12-15 19:58 UTC (permalink / raw
To: 38629, peter.ludemann
[-- Attachment #1: Type: text/plain, Size: 7084 bytes --]
I ran a large compilation (to the *compilation* buffer) (232,701 lines,
52M).
While running this, the memory usage increased as follows (output from ps
auwwwxxx):
Sun Dec 15 10:52:27 PST 2019
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
peter 1496 0.0 0.0 4524 828 pts/1 S 10:50 0:00
emacsclient -c
peter 31443 6.4 0.5 417868 90012 ? Ssl 10:49 0:10 emacs
--daemon
Sun Dec 15 11:42:41 PST 2019
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
peter 1496 0.0 0.0 4524 64 pts/1 S 10:50 0:00
emacsclient -c
peter 31443 5.6 51.9 9277928 8466908 ? Ssl 10:49 3:00 emacs
--daemon
I deleted the *compilation* window and ran garbage-collect; memory usage
decreased slightly, but not nearly to the original memory usage.
It appears that the *compilation* window doesn't get garbage collected.
Sun Dec 15 11:45:08 PST 2019
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
peter 1496 0.0 0.0 4524 64 pts/1 S 10:50 0:00
emacsclient -c
peter 31443 5.5 49.6 8913748 8103072 ? Ssl 10:49 3:03 emacs
--daemon
Running emacs daemon, started with the command
GDK=emacs emacs --daemon
In GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
of 2017-09-22, modified by Debian built on lgw01-amd64-050
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.3 LTS
Configured using:
'configure --build x86_64-linux-gnu --prefix=/usr
--sharedstatedir=/var/lib --libexecdir=/usr/lib
--localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.2/site-lisp:/usr/share/emacs/site-lisp
--with-sound=alsa --without-gconf --build x86_64-linux-gnu
--prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib
--localstatedir=/var/lib --infodir=/usr/share/info
--mandir=/usr/share/man --with-pop=yes
--enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.2/site-lisp:/usr/share/emacs/site-lisp
--with-sound=alsa --without-gconf --with-x=yes --with-x-toolkit=gtk3
--with-toolkit-scroll-bars 'CFLAGS=-g -O2
-fdebug-prefix-map=/build/emacs25-jYekUr/emacs25-25.2+1=.
-fstack-protector-strong
-Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
-D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''
Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11
Important settings:
value of $LC_MONETARY: en_CA.UTF-8
value of $LC_NUMERIC: en_CA.UTF-8
value of $LC_TIME: en_CA.UTF-8
value of $LANG: en_US.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix
Major mode: Shell
Minor modes in effect:
shell-dirtrack-mode: t
diff-auto-refine-mode: t
global-auto-revert-mode: t
show-paren-mode: t
display-time-mode: t
savehist-mode: t
desktop-save-mode: t
tooltip-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
column-number-mode: t
line-number-mode: t
transient-mark-mode: t
Recent messages:
Error during redisplay: (jit-lock-function 53777230) signaled (quit)
Quit
Mark set
Quit
Mark set
Quit [2 times]
Saving file /tmp/compilation-2...
Wrote /tmp/compilation-2
Quit [4 times]
Type C-x 1 to delete the help window.
Quit [5 times]
Load-path shadows:
/usr/share/emacs/25.2/site-lisp/debian-startup hides
/usr/share/emacs/site-lisp/debian-startup
/usr/share/emacs25/site-lisp/cmake-data/cmake-mode hides
/usr/share/emacs/site-lisp/cmake-mode
/usr/share/emacs/site-lisp/rst hides
/usr/share/emacs/25.2/lisp/textmodes/rst
~/emacs/prolog hides /usr/share/emacs/25.2/lisp/progmodes/prolog
/usr/share/emacs25/site-lisp/latex-cjk-thai/thai-word hides
/usr/share/emacs/25.2/lisp/language/thai-word
Features:
(shadow sort mail-extr emacsbug message rfc822 mml mml-sec epg mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mail-utils eieio-opt speedbar
sb-image ezimage dframe find-func css-mode misearch multi-isearch
minibuffer-complete-cycle server tempo erlang perl-mode asm-mode
cmake-mode conf-mode jka-compr add-log rst derived haskell-mode
haskell-cabal haskell-utils haskell-font-lock haskell-indentation
haskell-string haskell-sort-imports haskell-lexeme rx
haskell-align-imports haskell-compat haskell-complete-module
haskell-ghc-support flymake dabbrev haskell-customize go-mode url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap find-file ffap etags xref project make-mode
sh-script smie executable markdown-mode color url-parse url-vars
noutline outline tar-mode python tramp-sh tramp tramp-compat auth-source
cl-seq eieio eieio-core cl-macs gnus-util mm-util help-fns mail-prsvr
password-cache tramp-loaddefs trampver ucs-normalize format-spec
smerge-mode prolog align shell pcomplete dired vc-git diff-mode js
advice sgml-mode json map imenu thingatpt cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs finder-inf
go-mode-autoloads info package epg-config seq byte-opt gv bytecomp
byte-compile cl-extra help-mode easymenu cconv autorevert filenotify
grep compile comint ansi-color ring cus-start cus-load time-date paren
time savehist desktop frameset cl-loaddefs pcase cl-lib erlang-start
emacs-goodies-el emacs-goodies-custom emacs-goodies-loaddefs easy-mmode
devhelp mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)
Memory information:
((conses 16 637441 50039)
(symbols 48 38954 0)
(miscs 40 1139 1456)
(strings 32 101762 17154)
(string-bytes 1 3031599)
(vectors 16 64532)
(vector-slots 8 1858924 188630)
(floats 8 527 642)
(intervals 56 29869 3931)
(buffers 976 493))
[-- Attachment #2: Type: text/html, Size: 8104 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#38629: Loading/killing the compilation output doesn't lose memory
2019-12-15 19:58 bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Peter Ludemann
@ 2019-12-16 4:16 ` Peter Ludemann
2019-12-16 14:24 ` bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Noam Postavsky
1 sibling, 0 replies; 7+ messages in thread
From: Peter Ludemann @ 2019-12-16 4:16 UTC (permalink / raw
To: 38629
[-- Attachment #1: Type: text/plain, Size: 841 bytes --]
If I save the *compilation* buffer and restart emacs, memory usage goes up
modestly when I load the file (52MB; it is automatically fontified by emacs
when I load it):
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
peter 19037 0.6 0.7 441776 115476 ? Ssl 11:59 2:49 emacs
--daemon
peter 19037 0.6 1.0 495832 169776 ? Ssl 11:59 2:50 emacs
--daemon
and memory usage returns to about the previous value when I kill the buffer
(I don't need to run 'garbage-collect).
So, the problems seems to be somewhere in the "compile" command:
(a) it uses a *lot* more memory than needed to display the result
(b) it doesn't free that memory when compilation ends or when the
compilation buffer is killed
My "compilation" is actually compile plus 8200 tests, speeding them up by
using GNU-parallel.
[-- Attachment #2: Type: text/html, Size: 1290 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation*
2019-12-15 19:58 bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Peter Ludemann
2019-12-16 4:16 ` bug#38629: Loading/killing the compilation output doesn't lose memory Peter Ludemann
@ 2019-12-16 14:24 ` Noam Postavsky
2019-12-16 16:48 ` Peter Ludemann
2019-12-16 23:37 ` Peter Ludemann
1 sibling, 2 replies; 7+ messages in thread
From: Noam Postavsky @ 2019-12-16 14:24 UTC (permalink / raw
To: Peter Ludemann; +Cc: 38629
Peter Ludemann <peter.ludemann@gmail.com> writes:
> I ran a large compilation (to the *compilation* buffer) (232,701 lines,
> 52M).
> In GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
> of 2017-09-22, modified by Debian built on lgw01-amd64-050
> Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
> System Description: Ubuntu 18.04.3 LTS
I think this is a variant of Bug#26952 - "repeated buffer insertion
(e.g. yank-rectangle) consumes excessive memory (4GB+ for 90MB of
text)". I recommend upgrading to version 26.
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation*
2019-12-16 14:24 ` bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Noam Postavsky
@ 2019-12-16 16:48 ` Peter Ludemann
2019-12-16 23:37 ` Peter Ludemann
1 sibling, 0 replies; 7+ messages in thread
From: Peter Ludemann @ 2019-12-16 16:48 UTC (permalink / raw
To: Noam Postavsky; +Cc: 38629
[-- Attachment #1: Type: text/plain, Size: 855 bytes --]
It seems that emacs25 and emacs26 can't be in the same Ubuntu 18.04
installation (using ppa:kelleyk/emacs), so it'll take me a bit of work to
verify that the problem is fixed in emacs26.
On Mon, 16 Dec 2019 at 06:24, Noam Postavsky <npostavs@gmail.com> wrote:
> Peter Ludemann <peter.ludemann@gmail.com> writes:
>
> > I ran a large compilation (to the *compilation* buffer) (232,701 lines,
> > 52M).
>
> > In GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
> > of 2017-09-22, modified by Debian built on lgw01-amd64-050
> > Windowing system distributor 'The X.Org Foundation', version
> 11.0.11906000
> > System Description: Ubuntu 18.04.3 LTS
>
> I think this is a variant of Bug#26952 - "repeated buffer insertion
> (e.g. yank-rectangle) consumes excessive memory (4GB+ for 90MB of
> text)". I recommend upgrading to version 26.
>
>
>
[-- Attachment #2: Type: text/html, Size: 1440 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation*
2019-12-16 14:24 ` bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Noam Postavsky
2019-12-16 16:48 ` Peter Ludemann
@ 2019-12-16 23:37 ` Peter Ludemann
2019-12-17 0:19 ` Peter Ludemann
1 sibling, 1 reply; 7+ messages in thread
From: Peter Ludemann @ 2019-12-16 23:37 UTC (permalink / raw
To: Noam Postavsky; +Cc: 38629
[-- Attachment #1: Type: text/plain, Size: 1318 bytes --]
Upgrading to Emacs 26.3 seems to have fixed the memory problem but now
there's another problem ... emacs becomes incredibly sluggish (almost
unuseable) -- top(1) shows 70-100% CPU utilization even when the
compilation step isn't outputting anything. (I use GNU parallel on the
tests, with options to preserve the output order, so output happens at
intervals.)
The compilation command is "find ... | sort | nice parallel -j 8
--keep-order --group -L80" (on a 4 CPU machine).
I tried reducing the number of parallel jobs to 3, but that didn't help.
Should I open a new bug for this? If so, what information can I collect, to
help determine the cause?
On Mon, 16 Dec 2019 at 06:24, Noam Postavsky <npostavs@gmail.com> wrote:
> Peter Ludemann <peter.ludemann@gmail.com> writes:
>
> > I ran a large compilation (to the *compilation* buffer) (232,701 lines,
> > 52M).
>
> > In GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
> > of 2017-09-22, modified by Debian built on lgw01-amd64-050
> > Windowing system distributor 'The X.Org Foundation', version
> 11.0.11906000
> > System Description: Ubuntu 18.04.3 LTS
>
> I think this is a variant of Bug#26952 - "repeated buffer insertion
> (e.g. yank-rectangle) consumes excessive memory (4GB+ for 90MB of
> text)". I recommend upgrading to version 26.
>
>
>
[-- Attachment #2: Type: text/html, Size: 2207 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation*
2019-12-16 23:37 ` Peter Ludemann
@ 2019-12-17 0:19 ` Peter Ludemann
2019-12-17 13:14 ` Noam Postavsky
0 siblings, 1 reply; 7+ messages in thread
From: Peter Ludemann @ 2019-12-17 0:19 UTC (permalink / raw
To: Noam Postavsky; +Cc: 38629
[-- Attachment #1: Type: text/plain, Size: 238 bytes --]
The sluggishness is not related to the *compilation* window, so we can
close this bug (as fixed by emacs 26.3) and I'll open a new bug report.
Thank-you for your help! (And thank-you to Kevin Kelley for setting up the
PPA for emacs26.3)
[-- Attachment #2: Type: text/html, Size: 552 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation*
2019-12-17 0:19 ` Peter Ludemann
@ 2019-12-17 13:14 ` Noam Postavsky
0 siblings, 0 replies; 7+ messages in thread
From: Noam Postavsky @ 2019-12-17 13:14 UTC (permalink / raw
To: Peter Ludemann; +Cc: 38629
unarchive 26952
forcemerge 26952 38629
quit
Peter Ludemann <peter.ludemann@gmail.com> writes:
> The sluggishness is not related to the *compilation* window, so we can
> close this bug (as fixed by emacs 26.3) and I'll open a new bug report.
Thanks for checking.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-12-17 13:14 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-12-15 19:58 bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Peter Ludemann
2019-12-16 4:16 ` bug#38629: Loading/killing the compilation output doesn't lose memory Peter Ludemann
2019-12-16 14:24 ` bug#38629: 25.2; garbage-collect doesn't reclaim large *compilation* Noam Postavsky
2019-12-16 16:48 ` Peter Ludemann
2019-12-16 23:37 ` Peter Ludemann
2019-12-17 0:19 ` Peter Ludemann
2019-12-17 13:14 ` Noam Postavsky
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.