From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: When should ralloc.c be used? Date: Mon, 24 Oct 2016 09:27:43 -0700 Message-ID: <4f0c2868-d408-a5c4-d5a8-90dae750eb33@dancol.org> References: <7baa18d4-2b09-caa8-005e-29008a383ad1@cs.ucla.edu> <83mvhwrgd5.fsf@gnu.org> <8539f38f-9a11-44c3-4de7-bb974c96206c@cs.ucla.edu> <838ttfnmev.fsf@gnu.org> <837f8znk8f.fsf@gnu.org> <83zilvm2ud.fsf@gnu.org> <83r377m0i8.fsf@gnu.org> <83eg36n6v5.fsf@gnu.org> <83shrl523p.fsf@gnu.org> <83eg354ux3.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1477326511 28899 195.159.176.226 (24 Oct 2016 16:28:31 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 24 Oct 2016 16:28:31 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 Cc: eggert@cs.ucla.edu, emacs-devel@gnu.org To: Eli Zaretskii , Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Oct 24 18:28:27 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1byi6V-0004Ld-LD for ged-emacs-devel@m.gmane.org; Mon, 24 Oct 2016 18:28:04 +0200 Original-Received: from localhost ([::1]:47965 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1byi6X-0002nE-OR for ged-emacs-devel@m.gmane.org; Mon, 24 Oct 2016 12:28:05 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:34403) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1byi6K-0002lE-Fu for emacs-devel@gnu.org; Mon, 24 Oct 2016 12:27:53 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1byi6H-0003As-D1 for emacs-devel@gnu.org; Mon, 24 Oct 2016 12:27:52 -0400 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:38202) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1byi6G-0003AU-Mc; Mon, 24 Oct 2016 12:27:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:Cc:References:To:Subject; bh=8+5O4Y7B8oAXpgV+DOINfgm7dQXecOtweIl6uvmmTBM=; b=SFgxf8BOWNPkdQhjIt2tgLtAnM8KbCupUaTlsfXTUmNiVhqRwOUVAzsDmQDLMT5AxuchEdpdOn1dGO27NtbEhj/Ut5J/r/YwScmZP3GQLiREYZU4Tkrkv3SqF9VjLKATC+xtVUsdyV0JeoPGh/5vLcURhWJcaJ78i/1fXVAw8EEsIAnz09MKQJ8iywWxAluvqAlQl3WJwuYr5txaz8ya9HybTkpL3AawFg1ED15zvxI93vTPw/CdvyLQaWGQKmMkW/BSLNAPa4wcFAXcFRQ+xi4be+QG+1sgyz9JCiv6yD3B28O7rPeObu01ggdlF3NpIEz2u40isOz9rofYXgVp1w==; Original-Received: from c-73-97-199-232.hsd1.wa.comcast.net ([73.97.199.232] helo=[192.168.1.173]) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1byi6F-0004bC-GI; Mon, 24 Oct 2016 09:27:47 -0700 In-Reply-To: <83eg354ux3.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208701 Archived-At: On 10/24/2016 08:40 AM, Eli Zaretskii wrote: >> From: Stefan Monnier >> Cc: eggert@cs.ucla.edu, emacs-devel@gnu.org >> Date: Mon, 24 Oct 2016 10:37:19 -0400 >> >>> Using mmap has disadvantages: when you need to enlarge buffer text, >>> and that fails (because there are no more free pages/addresses after >>> the already allocated region), we need to copy buffer text to the new >>> allocation. 64-bit address spaces are *huge*. What about just making every buffer allocation 2GB long or so, marked PROT_NONE? You don't actually have to commit all that memory --- all you've done is set aside that address space. But because you've set aside so much address space, you'll very likely be able to expand the actual allocation region (a subset of the reserved region) as much as you want. >> All allocators suffer from this problem. I haven't seen any evidence >> that the mmap-based allocation code is significantly more prone to it. > > I have seen that. The native glibc malloc, the on GNU/Linux systems > were using until we got screwed by the recent glibc, didn't have this > problem, while mmap-based allocator did. Don't ask me how glibc does > it, I don't know; but the fact is there. This was discovered when the > Windows mmap emulation in w32heap.c was developed and tested. > >> Also, the glibc allocators used mmap internally when allocating >> large-ish chunks (e.g. for buffer text), so if that was a problem, we >> would have noticed, I think. > > True; but they somehow work around the problem. > >> Another advantage of using mmap is that it can return the memory to the >> OS once you kill your large buffer, whereas with gmalloc+ralloc this >> basically never happens, AFAIK. > > Not entirely true: ralloc calls the system sbrk with a negative > argument when it feels like it. You can also madvise(MADV_DONTNEED, ...) regions *inside* the heap that contain only freed memory. This procedure also returns memory to the operating system.