From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: Re: When should ralloc.c be used? Date: Wed, 26 Oct 2016 00:36:42 -0400 Message-ID: <9F4BB1CC-0945-4254-94A0-DCEC30A09815@raeburn.org> References: <831szqhbc2.fsf@gnu.org> <87d1itt79z.fsf_-_@users.sourceforge.net> <7baa18d4-2b09-caa8-005e-29008a383ad1@cs.ucla.edu> <83mvhwrgd5.fsf@gnu.org> <8539f38f-9a11-44c3-4de7-bb974c96206c@cs.ucla.edu> <838ttfnmev.fsf@gnu.org> <837f8znk8f.fsf@gnu.org> <83zilvm2ud.fsf@gnu.org> <83r377m0i8.fsf@gnu.org> <83eg36n6v5.fsf@gnu.org> <83shrl523p.fsf@gnu.org> <83oa28wgzb.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1477456628 1051 195.159.176.226 (26 Oct 2016 04:37:08 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 26 Oct 2016 04:37:08 +0000 (UTC) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Oct 26 06:37:04 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bzFxQ-000798-4M for ged-emacs-devel@m.gmane.org; Wed, 26 Oct 2016 06:36:56 +0200 Original-Received: from localhost ([::1]:59563 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzFxS-0007bL-93 for ged-emacs-devel@m.gmane.org; Wed, 26 Oct 2016 00:36:58 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:42923) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzFxK-0007aW-I4 for emacs-devel@gnu.org; Wed, 26 Oct 2016 00:36:51 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bzFxG-0000o7-6p for emacs-devel@gnu.org; Wed, 26 Oct 2016 00:36:50 -0400 Original-Received: from mail-qt0-x229.google.com ([2607:f8b0:400d:c0d::229]:36440) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1bzFxF-0000lD-VA for emacs-devel@gnu.org; Wed, 26 Oct 2016 00:36:46 -0400 Original-Received: by mail-qt0-x229.google.com with SMTP id 12so246314qtm.3 for ; Tue, 25 Oct 2016 21:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raeburn-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3b/ccffL//i3BhPotGDGE3KbOzoyNBfHJBReE6ApFg4=; b=0N9+lXkN833wjqXPckbI7VoOPZBkVQHwvatAaMB8G/m601igeawl0zQngWWJBGwrLk WFODT/SE2fKhyXR7wUDQZZpgALCJf4Job1XnFOwCZ7diNX1lpgwzUCYx2PCIdynUyTHK wVhWQq7Gn3HVe5/34CVB25jcPAPkKqalHE+8g+zzLwCIqS3nrUHxfehiv9066ik6H4yW NW0pIgRxJPTbD1z6AkO4QI8KeWMUM3lLVPujRvQLHOHwmDV+a85EEat4QoEO3CkWeqo/ mU++r1CUaesn7q5PueRTRhnnM4FZ0D3VLwi15GGOdVuEz1WpjjPondw9PrUC3twD4S80 66Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3b/ccffL//i3BhPotGDGE3KbOzoyNBfHJBReE6ApFg4=; b=TyqZFmf1lFXBVk8y5QqmPZ7gMsmxVVN6ZV4PrRU9NDijLNaoyxZnKuk3lmptlTkdP/ fayFgC+h0pbDh2eRkjlhw9f0aEFp6CF3im5jHVNEAwnR+EOwWjG3q+PeqKGnSF9pO1Gx Aj4ojUjjE8odo2mdypH3apBIahtzoSe2pbe86HSpKFNS6KbWNZGq65M/+veZtkXcTjA+ bEG2xojnQRGvsMjdSX9npLggbfsZELd6M438W1kydJ6pOgyuKfgEN8ThzOtAFSVhwhKF DjorRIztRg+CPOr4Wx88upQczK7OsjPdUJm9So97mMopJnEaZ0F3T85y4UmvsGazuBXH vfYg== X-Gm-Message-State: ABUngvcnaZJHZc+DgewmCxmf8rS1TkQqh+rUOefI3UJtL/pmwOyNP8i3gCZQbz89iPwMYg== X-Received: by 10.237.54.98 with SMTP id e89mr4373578qtb.130.1477456604191; Tue, 25 Oct 2016 21:36:44 -0700 (PDT) Original-Received: from [192.168.23.52] (c-50-138-183-136.hsd1.ma.comcast.net. [50.138.183.136]) by smtp.gmail.com with ESMTPSA id g143sm98331qke.18.2016.10.25.21.36.43 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 25 Oct 2016 21:36:43 -0700 (PDT) In-Reply-To: <83oa28wgzb.fsf@gnu.org> X-Mailer: Apple Mail (2.3124) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400d:c0d::229 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208820 Archived-At: > On Oct 25, 2016, at 12:06, Eli Zaretskii wrote: >=20 >> From: Ken Raeburn >> Date: Mon, 24 Oct 2016 23:12:40 -0400 >>=20 >>> Using mmap has disadvantages: when you need to enlarge buffer text, >>> and that fails (because there are no more free pages/addresses after >>> the already allocated region), we need to copy buffer text to the = new >>> allocation. This happens quite a lot when we visit a compressed >>> buffer. (The MS-Windows emulation of mmap in w32heap.c reserves = twice >>> the number of pages as originally requested, for that very reason.) >>=20 >> In the general case, yes. But modern Linux kernels have an = =E2=80=9Cmremap=E2=80=9D system=20 >> call which can =E2=80=9Cmove=E2=80=9D a range of pages to a portion = of the address space that=20 >> can accommodate a larger size, by tweaking page tables rather than = copying all=20 >> the bits around. I=E2=80=99m pretty sure modern glibc realloc uses = it. >=20 > AFAIU, this feature will only help us if someone adds code to use it > in buffer.c:mmap_enlarge. Or are you saying that the OS will call > mremap for us automatically when mmap_enlarge attempts to map > additional pages at the end of an mmaped region? It could be done explicitly, but my experience was that malloc/realloc = would just do it for us; we=E2=80=99d just have to use malloc/realloc = instead of explicitly calling mmap. I just took a quick look at the = glibc sources (2.19, as patched and packaged by Debian), and it looks = like the use of mmap kicks in by default for 128kB or larger = allocations, though the threshold can be changed at run time. > If the Linux kernel is the only system that allows implementation of > mremap, then it doesn't really help in the long run, because on master > we don't need mmap at all for GNU/Linux systems. A man page browser at freebsd.org for several platforms seems to = indicate that NetBSD has picked it up, but neither FreeBSD nor OpenBSD. = I don=E2=80=99t know if NetBSD=E2=80=99s realloc will use it, but it=E2=80= =99s certainly simpler if we just ignore mremap for explicit use, and = just bear in mind that realloc may not always have to pay the expected = copying penalty on all systems=E2=80=A6.=