Re: Memory again

[Stefan Monnier <monnier@iro.umontreal.ca>]

> But right now, having had emacs up for a few days, and only opening 10
> small files with the aid of emacsclient, emacs's RSS is 130M.  It had
> climbed up to 400M before I most recently killed it.

400M definitely sounds like a bug.

> What's the differences between Emacs's and XEmacs's allocators?

I don't think the source of your problem/difference is in the allocator.

> By the way, the dismissal of this being a real problem because Emacs can
> always reuse the fragmented memory either

I don't really dismiss it.  My point is rather the following:
- Many memory problems are misdiagnosed as fragmentation problems, where
  the real cause is either inefficient memory use, or memory leaks.
- Real fragmentation problems exist but are fairly rare.
- In some "fragmentation" cases, the Emacs process holds on to memory
  without any good reason, i.e. it's been "free"d but the malloc library
  does not want or can't return it to the OS because it did not use mmap
  to allocate it.  This can be fixed, but this memory would really be
  unused; it can still appear in RSS but only because the memory
  pressure hasn't yet bumped it out of RAM.  IOW these cases may show
  high memory use in terms of VSZ or RSS but fixing them is low priority
  because their only direct cost is use of of swap space.
- Fixing the remaining real fragmentation problems probably requires
  a different allocator that can move objects to compact the memory
  space.  Maybe such an allocator can be retrofitted into Emacs
  (e.g. a mostly-copying allocator), but noone has tried it yet
  (probably because the outcome is hard to predict, the problem it
  attempts to fix only occurs rather rarely, and it can be difficult to
  ensure that it doesn't affect negatively the more common cases).

So, I know such problems exist in real life, and I would be happy to
accept patches that fix them, but it's not very high on my list
of priorities.


        Stefan