From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: jca@wxcvbn.org (=?utf-8?Q?J=C3=A9r=C3=A9mie_Courr=C3=A8ges-Anglas?=) Newsgroups: gmane.emacs.devel Subject: Re: When should ralloc.c be used? Date: Fri, 28 Oct 2016 13:40:10 +0200 Message-ID: <87vawcem79.fsf@ritchie.wxcvbn.org> References: <837f8znk8f.fsf@gnu.org> <83zilvm2ud.fsf@gnu.org> <83r377m0i8.fsf@gnu.org> <83eg36n6v5.fsf@gnu.org> <83shrl523p.fsf@gnu.org> <83eg354ux3.fsf@gnu.org> <4f0c2868-d408-a5c4-d5a8-90dae750eb33@dancol.org> <878tt9ggdk.fsf@ritchie.wxcvbn.org> <83k2cssypt.fsf@gnu.org> <6350b2df-fde9-e716-d279-9f29438f8ee5@dancol.org> <83d1ikswsf.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1477654955 16033 195.159.176.226 (28 Oct 2016 11:42:35 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 28 Oct 2016 11:42:35 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (berkeley-unix) Cc: Daniel Colascione , emacs-devel@gnu.org, monnier@iro.umontreal.ca, eggert@cs.ucla.edu To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Oct 28 13:42:30 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c05YG-0002oB-O5 for ged-emacs-devel@m.gmane.org; Fri, 28 Oct 2016 13:42:24 +0200 Original-Received: from localhost ([::1]:48401 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c05YJ-0006tE-7x for ged-emacs-devel@m.gmane.org; Fri, 28 Oct 2016 07:42:27 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51985) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c05YA-0006sy-OB for emacs-devel@gnu.org; Fri, 28 Oct 2016 07:42:19 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c05Y9-0003qr-QH for emacs-devel@gnu.org; Fri, 28 Oct 2016 07:42:18 -0400 Original-Received: from chomsky.autogeree.net ([91.216.110.36]:59864) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c05Y5-0003pq-UV; Fri, 28 Oct 2016 07:42:14 -0400 Original-Received: from ritchie.wxcvbn.org (localhost [127.0.0.1]) by wxcvbn.org (8.15.2/8.15.2) with ESMTP id u9SBeFQc055572; Fri, 28 Oct 2016 13:40:15 +0200 (CEST) Original-Received: (from jca@localhost) by ritchie.wxcvbn.org (8.15.2/8.15.2/Submit) id u9SBeAw6039461; Fri, 28 Oct 2016 13:40:10 +0200 (CEST) Mail-Followup-To: Eli Zaretskii , Daniel Colascione , eggert@cs.ucla.edu, monnier@iro.umontreal.ca, emacs-devel@gnu.org In-Reply-To: <83d1ikswsf.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 28 Oct 2016 11:27:44 +0300") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 91.216.110.36 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208936 Archived-At: Eli Zaretskii writes: >> Cc: monnier@iro.umontreal.ca, eggert@cs.ucla.edu, emacs-devel@gnu.org >> From: Daniel Colascione >> Date: Fri, 28 Oct 2016 01:11:08 -0700 >> >> Say I mmap (anonymously, for simplicity) a page PROT_NONE. After the >> initial mapping, that address space is unavailable for other uses. But >> because the page protections are PROT_NONE, my program has no legal >> right to access that page, so the OS doesn't have to guarantee that it ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> can find a physical page to back that page I've mmaped. In this state, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is what I think is a problem in your reasoning. "Doesn't have to guarantee" doesn't mean that the kernel *should not* actually check the available memory and resource limits. >> the memory is reserved. >> >> The 20GB PROT_NONE address space reservation itself requires very little >> memory. It's just a note in the kernel's VM interval tree that says "the >> addresses in range [0x20000, 0x500020000) are reserved". Virtual memory is >> >> Now imagine I change the protections to PROT_READ|PROT_WRITE --- once >> the PROT_READ|PROT_WRITE mprotect succeeds, my program has every right >> to access that page; under a strict accounting scheme (that is, without >> overcommit), the OS has to guarantee that it'll be able to go find a >> physical page to back that virtual page. In this state, the memory is >> committed -- the kernel has committed to finding backing storage for >> that page at some point when the current process tries to access it. > > I'm with you up to here. My question is whether PROT_READ|PROT_WRITE > call could fail after PROT_NONE succeeded. You seem to say it could; > I thought it couldn't. I wouldn't have thought that PROT_NONE vs PROT_READ|PROT_WRITE would have changed anything here, but on *some* OSes it does, however it is not portable. At least OpenBSD doesn't behave like what you describe. IMHO people who rely on this kind of reservations rely on implementation-defined behavior. Also, sanity wise, I'd prefer having mmap(2) fail right away rather than having mprotect(2) fail, much later. *If* mprotect(2) actually fails ; of course, you don't want to play russian roulette with your OS's flavor of the OOM-killer either. >> Say you have a strict-accounting system with 1GB of RAM and 1GB of swap. >> I can write a program that reserves 20GB of address space. > > I thought such a reservation should fail, because you don't have > enough virtual memory for 20GB of addresses. IOW, I thought the > ability to reserve address space is restricted by the actual amount of > virtual memory available on the system at the time of the call. You > seem to say I was wrong. -- jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE