Re: the ...-unload-hook convention doesn't work

[Dave Love <d.love@dl.ac.uk>]

Simon Josefsson <jas@extundo.com> writes:

> Which feature is this?

gnus-nocem (though it's probably not be doing me much good anyway
without reading usenet).

> Any hints on finding leaking code, in general?

In that case I guess it fails to purge a hash table, which I couldn't
easily follow.

> I have been using the following to print out large symbols:
>
> (mapatoms (lambda (sym)
> 	    (let ((len (ignore-errors (length (symbol-value sym)))))
> 	      (if (and len (> len 500) (not (= len 507904)))
> 		  (insert (format "%s: %s\n" sym len))))))
>
> But it doesn't seem to work well.

You need dependency info tracing all slots from obarrays to start
with, and then you won't account for data kept live through C code, at
least (e.g. text properties).

> I have been getting complaints
> about exhausting the memory from Emacs without that code generating
> anything interesting.  My machine has 1 GB RAM, and emacs is typically
> around 100MB but can grow to 300MB or so when I do something major in
> Gnus.

The message is probably misleading in that case -- see all its uses in
alloc.c (?).

The limit on what a Lisp pointer can address is 128MB (or 256 -- I
can't remember) on a 32-bit system.  I added a comment to that effect
somewhere.  The actual data in strings, for instance, aren't
necessarily in the same address space, but as far as I remember,
probably contribute to that limit because they're malloced from the
same region of (end of) the machine's address space (whereas mmapped
space is at the other end, at least under Linux).  See (probably) the
same comment as above.

It's made worse because pages allocated in the big bag don't get freed
at least until they're completely emptied by GC -- see unused conses
&c in the GC statistics.

(A different sort of leak -- buffer space -- can occur on systems like
Solaris which don't use the relocating allocator, and/or the mmap
mechanism -- I forget.  I don't remember whether the buffer space is
in the Lisp pointer range in that case or not.  I thought rms vetoed
the solution, but it's still in the version of TODO I have).

> Essentially: how do I find out what is taking up all memory in Emacs?
> Perhaps a package that report these things would be handy.

You can't, without serious work on the internals.

Replacing the GC, and perhaps the object representation, per TODO
could/would address all these issues to a greater or lesser extent.  I
assume there's a policy against that now since the old one is being
tinkered with.  Implementing the TODO item on `long long' would remove
the addressing limitation at the expense of larger data, at least
using GCC.