Re: Proposal: block-based vector allocator

[Dmitry Antipov <dmantipov@yandex.ru>]

On 12/08/2011 06:10 PM, Stefan Monnier wrote:

> Someone could try to change the allocator so those blocks are exactly
> 1-page in size and get allocated via mmap:

I did this even before I started with block-based vector allocator.

> - using mmap will ensure that we factor out malloc from the equation.
> - using page-size (presumably 4KB) blocks rather than the current 16KB

Current sizes are 1K except string data, which is 8K.

> Then re-run some problematic case (e.g. the million-lines compile buffer)
> to see if this helps.  It really should help, tho I don't hold much hope
> that it will make a significant difference (e.g. more than 30%
> difference would be significant).

My favorite byte-force-recompile benchmark usually consumes ~89M. With my
vector allocator it shrinks to 78M. When mmap() is enabled, it shrinks
to 76M. Not too much.

But for (definitely pathological) compile buffer case, results are
very unexpected (I tried with 100000-lines buffer). Fresh instance
is 18M. With compile buffer, it grows to 142M and shrinks to 141M
after killing *compilation* and garbage-collect. When mmap() is enabled,
it grows to 138M and shrinks back to 21M (twenty one) after kill & GC.

Dmitry