From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Dmitry Antipov Newsgroups: gmane.emacs.devel Subject: Re: Proposal: block-based vector allocator Date: Fri, 09 Dec 2011 13:04:09 +0400 Message-ID: <4EE1CF09.70700@yandex.ru> References: <4EDDA68B.5050601@yandex.ru> <4EDE315E.3030804@yandex.ru> <4EDEF421.6090800@yandex.ru> <4EDF8F8E.8050401@yandex.ru> <87hb1b25i7.fsf@uwakimon.sk.tsukuba.ac.jp> <4EE03FFD.1090400@yandex.ru> <4EE0EA5C.1090505@yandex.ru> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1323421458 7023 80.91.229.12 (9 Dec 2011 09:04:18 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 9 Dec 2011 09:04:18 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Dec 09 10:04:14 2011 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RYwNH-0005TO-Cn for ged-emacs-devel@m.gmane.org; Fri, 09 Dec 2011 10:04:11 +0100 Original-Received: from localhost ([::1]:40800 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RYwNG-0005eY-VT for ged-emacs-devel@m.gmane.org; Fri, 09 Dec 2011 04:04:10 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:49968) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RYwNE-0005eM-KP for emacs-devel@gnu.org; Fri, 09 Dec 2011 04:04:09 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RYwNC-0000Ag-Ux for emacs-devel@gnu.org; Fri, 09 Dec 2011 04:04:08 -0500 Original-Received: from mail.dev.rtsoft.ru ([213.79.90.226]:36730) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1RYwNC-00009f-Ca for emacs-devel@gnu.org; Fri, 09 Dec 2011 04:04:06 -0500 Original-Received: (qmail 5115 invoked from network); 9 Dec 2011 09:04:03 -0000 Original-Received: from unknown (HELO ?192.168.5.146?) (192.168.1.70) by 0 with SMTP; 9 Dec 2011 09:04:03 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111115 Thunderbird/8.0 In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Received-From: 213.79.90.226 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:146594 Archived-At: On 12/08/2011 11:58 PM, Stefan Monnier wrote: > How much did it help? Not too much because it makes no sense without block-based vector allocation - huge amounts of small vectors and corresponding mem_nodes allocated from brk()ed heap devastates everything. And it was a bit slower, most probably because explicit mmmap() implies zeroing performed by OS. > And you said that using the non-conservative stack scanning resulted in > similar results, so all that extra memory is occupied by mem_nodes (or > fragments that used to hole mem_nodes ;-). Yes, one mem_node per each vector results in substantial overhead and fragmentation. _Any_ block-based vector allocator should reduce the number of mem_nodes, so saving space - but rounding allocator will consume some of this space back for it's own internal fragmentation. That's why I proposed the scheme with tightest possible fit instead of 'standard malloc algorithm'. Even for non-conservative stack scanning with eliminated mem_nodes overhead, I'm pretty sure that my proposal works at least not worse, taking both CPU and memory usage into account; rounding vector allocator will consume more memory, but may have near to negligible advantage in CPU usage because it has no splitting/coalescing overhead. Dmitry