From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: When should ralloc.c be used? Date: Fri, 28 Oct 2016 08:34:53 -0700 Message-ID: <7feaba5a-df5d-9d78-be0a-15756cbee898@dancol.org> References: <837f8znk8f.fsf@gnu.org> <83zilvm2ud.fsf@gnu.org> <83r377m0i8.fsf@gnu.org> <83eg36n6v5.fsf@gnu.org> <83shrl523p.fsf@gnu.org> <83eg354ux3.fsf@gnu.org> <4f0c2868-d408-a5c4-d5a8-90dae750eb33@dancol.org> <878tt9ggdk.fsf@ritchie.wxcvbn.org> <83k2cssypt.fsf@gnu.org> <6350b2df-fde9-e716-d279-9f29438f8ee5@dancol.org> <83d1ikswsf.fsf@gnu.org> <87vawcem79.fsf@ritchie.wxcvbn.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1477670586 26350 195.159.176.226 (28 Oct 2016 16:03:06 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 28 Oct 2016 16:03:06 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 To: Eli Zaretskii , eggert@cs.ucla.edu, monnier@iro.umontreal.ca, emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Oct 28 18:02:58 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c09cH-0004er-9t for ged-emacs-devel@m.gmane.org; Fri, 28 Oct 2016 18:02:49 +0200 Original-Received: from localhost ([::1]:50038 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c09cJ-0008RM-PS for ged-emacs-devel@m.gmane.org; Fri, 28 Oct 2016 12:02:51 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:59968) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c09BQ-0001mD-NV for emacs-devel@gnu.org; Fri, 28 Oct 2016 11:35:05 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c09BM-0003Io-P1 for emacs-devel@gnu.org; Fri, 28 Oct 2016 11:35:04 -0400 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:49018) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1c09BM-0003IW-Fz; Fri, 28 Oct 2016 11:35:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject; bh=jW34f1CO+43LutPhyHnddod6xybNl+plDgj1mwEUmKY=; b=b9r3OCQ0MoMg6UIr6tlg+vP8c8qjED09daIGckmJctmr7p+310TeJ/0P6QxyU8g1lsUuTC7Ge+AAQ1smApyK+XdsW3K4F6TzfuaKEFI3goaA2NZtHj7sIbOpaWf5GXYp0MCLulSl4HL4+PkiTAqDtjC076XXr5uk5llGMHGuSAqynlZ+Gj7b409DIbZW1pBgi1UlSbMjaRrOgo99QtH8yupr8OA0xc/4zDlf0HIaIuO3ioyslf080X54bUDMya4r9URruUKdV+oM1PTHfNBLZDer1tWelSgycJocBc1+8hGc+RNPwI26YVbVzzcTmjZCxJQAerRyftH4oujwnc0Vaw==; Original-Received: from c-73-97-199-232.hsd1.wa.comcast.net ([73.97.199.232] helo=[192.168.1.173]) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1c09BJ-00082r-QI; Fri, 28 Oct 2016 08:34:57 -0700 In-Reply-To: <87vawcem79.fsf@ritchie.wxcvbn.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208949 Archived-At: On 10/28/2016 04:40 AM, Jérémie Courrèges-Anglas wrote: > Eli Zaretskii writes: > >>> Cc: monnier@iro.umontreal.ca, eggert@cs.ucla.edu, emacs-devel@gnu.org >>> From: Daniel Colascione >>> Date: Fri, 28 Oct 2016 01:11:08 -0700 >>> >>> Say I mmap (anonymously, for simplicity) a page PROT_NONE. After the >>> initial mapping, that address space is unavailable for other uses. But >>> because the page protections are PROT_NONE, my program has no legal >>> right to access that page, so the OS doesn't have to guarantee that it > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> can find a physical page to back that page I've mmaped. In this state, > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > This is what I think is a problem in your reasoning. "Doesn't have to > guarantee" doesn't mean that the kernel *should not* actually check the > available memory and resource limits. > IMHO, an OS that rejects big PROT_NONE mappings merely because it might not be able to change them to PROT_READ|PROT_WRITE later is broken. The non-overcommit Linux behavior (which is identical to Windows behavior) is the _right _thing_ _to_ _do_. The OS is letting the process manage its address space and assuming that the programmer knows what he wanted to do. >>> the memory is reserved. >>> >>> The 20GB PROT_NONE address space reservation itself requires very little >>> memory. It's just a note in the kernel's VM interval tree that says "the >>> addresses in range [0x20000, 0x500020000) are reserved". Virtual memory is >>> >>> Now imagine I change the protections to PROT_READ|PROT_WRITE --- once >>> the PROT_READ|PROT_WRITE mprotect succeeds, my program has every right >>> to access that page; under a strict accounting scheme (that is, without >>> overcommit), the OS has to guarantee that it'll be able to go find a >>> physical page to back that virtual page. In this state, the memory is >>> committed -- the kernel has committed to finding backing storage for >>> that page at some point when the current process tries to access it. >> >> I'm with you up to here. My question is whether PROT_READ|PROT_WRITE >> call could fail after PROT_NONE succeeded. You seem to say it could; >> I thought it couldn't. > > I wouldn't have thought that PROT_NONE vs PROT_READ|PROT_WRITE would > have changed anything here, but on *some* OSes it does, however it is > not portable. At least OpenBSD doesn't behave like what you describe. How does it behave? > IMHO people who rely on this kind of reservations rely on > implementation-defined behavior. OpenBSD is a Coelacanth. It's a relic. It doesn't even a unified buffe cache. > Also, sanity wise, I'd prefer having mmap(2) fail right away rather than > having mprotect(2) fail, much later. Then ask for PROT_READ|PROT_WRITE access right away. Ask for commit, not just address space. > *If* mprotect(2) actually fails ; > of course, you don't want to play russian roulette with your OS's > flavor of the OOM-killer either. That's why overcommit is an abomination.