From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: When should ralloc.c be used? Date: Wed, 26 Oct 2016 14:40:33 +0300 Message-ID: <83lgxbuymm.fsf@gnu.org> References: <831szqhbc2.fsf@gnu.org> <87d1itt79z.fsf_-_@users.sourceforge.net> <7baa18d4-2b09-caa8-005e-29008a383ad1@cs.ucla.edu> <83mvhwrgd5.fsf@gnu.org> <8539f38f-9a11-44c3-4de7-bb974c96206c@cs.ucla.edu> <838ttfnmev.fsf@gnu.org> <837f8znk8f.fsf@gnu.org> <83zilvm2ud.fsf@gnu.org> <83r377m0i8.fsf@gnu.org> <83eg36n6v5.fsf@gnu.org> <83shrl523p.fsf@gnu.org> <83oa28wgzb.fsf@gnu.org> <9F4BB1CC-0945-4254-94A0-DCEC30A09815@raeburn.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1477482097 14537 195.159.176.226 (26 Oct 2016 11:41:37 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 26 Oct 2016 11:41:37 +0000 (UTC) Cc: emacs-devel@gnu.org To: Ken Raeburn Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Oct 26 13:41:31 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bzMaF-0002bh-5Y for ged-emacs-devel@m.gmane.org; Wed, 26 Oct 2016 13:41:27 +0200 Original-Received: from localhost ([::1]:33431 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzMaH-0005Up-Lu for ged-emacs-devel@m.gmane.org; Wed, 26 Oct 2016 07:41:29 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40783) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzMZS-0005RV-49 for emacs-devel@gnu.org; Wed, 26 Oct 2016 07:40:42 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bzMZN-00036y-6J for emacs-devel@gnu.org; Wed, 26 Oct 2016 07:40:38 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33083) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzMZN-00036u-2v; Wed, 26 Oct 2016 07:40:33 -0400 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2049 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bzMZM-0003Ki-FB; Wed, 26 Oct 2016 07:40:32 -0400 In-reply-to: <9F4BB1CC-0945-4254-94A0-DCEC30A09815@raeburn.org> (message from Ken Raeburn on Wed, 26 Oct 2016 00:36:42 -0400) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208831 Archived-At: > From: Ken Raeburn > Date: Wed, 26 Oct 2016 00:36:42 -0400 > Cc: emacs-devel@gnu.org > > >>> Using mmap has disadvantages: when you need to enlarge buffer text, > >>> and that fails (because there are no more free pages/addresses after > >>> the already allocated region), we need to copy buffer text to the new > >>> allocation. This happens quite a lot when we visit a compressed > >>> buffer. (The MS-Windows emulation of mmap in w32heap.c reserves twice > >>> the number of pages as originally requested, for that very reason.) > >> > >> In the general case, yes. But modern Linux kernels have an “mremap” system > >> call which can “move” a range of pages to a portion of the address space > >> that > >> can accommodate a larger size, by tweaking page tables rather than copying > >> all > >> the bits around. I’m pretty sure modern glibc realloc uses it. > > > > AFAIU, this feature will only help us if someone adds code to use it > > in buffer.c:mmap_enlarge. Or are you saying that the OS will call > > mremap for us automatically when mmap_enlarge attempts to map > > additional pages at the end of an mmaped region? > > It could be done explicitly, but my experience was that malloc/realloc would > just do it for us; we’d just have to use malloc/realloc instead of explicitly > calling mmap. I think we've lost context of the discussion. Please see above: this is about the disadvantages of using mmap directly, i.e. for those cases where the native malloc or gmalloc suffer from memory fragmentation, and we decide to use mmap in buffer.c to countermand that. I've pointed out the disadvantages of using mmap directly, and you mentioned the mremap syscall as the counter-argument. If you thought I was talking about problems mmap could cause to the malloc implementation, then that's a misunderstanding: I was explicitly talking about using mmap directly for allocating buffer text. My point was that we should only use mmap if necessary, as it comes for a price. > A man page browser at freebsd.org for several platforms seems to indicate that > NetBSD has picked it up, but neither FreeBSD nor OpenBSD. I don’t know if > NetBSD’s realloc will use it, but it’s certainly simpler if we just ignore > mremap for explicit use, and just bear in mind that realloc may not always have > to pay the expected copying penalty on all systems…. Once again, this is about the cases where using malloc for buffer text gives unsatisfactory results, and mmap is being considered as a remedy.