Re: why is emacs using all my memory

[pjb@informatimago.com (Pascal J. Bourguignon)]

David Pleydell <d_pleydell@yahoo.com> writes:

> I have writen a program in C (called from a script in R) that I run
> inside emacs. The program is still in development phase so I have it
> spit out lots and lots and lots of text for monitorring progress. The
> program runs for several days. 
>
> After half a day emacs is now using over 3Gb of RAM (almost half of my
> RAM). So I killed all the old text and purged the kill ring using
> M-:(setq kill-ring nil). But this hasn't freed the 3Gb. I have no idea
> what emacs is doing with this 3Gb, there are no other processes
> running. How can I diagnose what process within emacs is using this
> memory? Perhaps there is there some kind of "M-x top"?

Better than setting kill-ring to nil, I think you should use
(buffer-disable-undo).  

And then copy a character or two again, to free the old undo stuff.

You may also want to set comint-buffer-maximum-size.

But foremost, I would advise you to use redirection to a file, and use
from time to time tail or tail -f.

cmd > /tmp/big.output & 
tail -40 /tmp/big.output
tail -f /tmp/big.output & pid=$? ; sleep 60 ; kill $?


If there are easily identified lines that are ignorable, grep -v may
also come handy:

cmd | grep -v 'do not care' > /tmp/medium.output &
tail -f /tmp/medium.output 


Yes, it's unfortunate that it requires some pre-thinking.
The alternative would be to design a shell that would automatically
insert redirection points (ptys), and that would stop outputing to the
terminal when output is too big, but store it automatically to a spool
file.  Then, in a TTY, it could display transiently the current lines,
but in a dumb terminal (such as emacs shell), it would not output
anything unless requested by the user (eg. skipping directly to the end, or
wherever the user wants à la less(1)).

What's more, a mechanism to distinguish important messages from dumb
logs would allow such a shell to keep a list of important message to be
read and acknowledged, while the dumb logs would just fill the spool and
be of use only in case of problem.


On the old systems of lore, there were such "shells", to manage the
messages directed to the operators.

-- 
__Pascal Bourguignon__
http://www.informatimago.com