From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Preview: portable dumper Date: Mon, 19 Feb 2018 09:03:38 -0800 Message-ID: References: <797cf77a-337e-459a-a82b-28d2c63418a7@email.android.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1519059784 9999 195.159.176.226 (19 Feb 2018 17:03:04 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 19 Feb 2018 17:03:04 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 Cc: Eli Zaretskii , emacs-devel@gnu.org To: Ken Brown Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Feb 19 18:02:59 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1enopz-0001PM-5i for ged-emacs-devel@m.gmane.org; Mon, 19 Feb 2018 18:02:47 +0100 Original-Received: from localhost ([::1]:51368 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1enorz-0000Nl-Nx for ged-emacs-devel@m.gmane.org; Mon, 19 Feb 2018 12:04:51 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51196) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1enor4-0008Hd-B0 for emacs-devel@gnu.org; Mon, 19 Feb 2018 12:03:55 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1enor3-0000SO-1A for emacs-devel@gnu.org; Mon, 19 Feb 2018 12:03:54 -0500 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:52132) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1enor2-0000NW-Oj; Mon, 19 Feb 2018 12:03:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject; bh=t1cPFajwxUgtnl2McBEfubgpYaL0BnT3RqxrWGMCBDo=; b=oOE+8I4pYH4rr6AGyNDiAVLTOVOAa07k44J6dK/qWhvUzLraW10bna2LTf/fOVI8ZaieqYwGH3KCuWa/0njrWR3vlNCR1+jPMffBMR9hEyEgnbNzJvalocHUq+C+S/L0S0TKIH4hWJHSpfU+vQCqxCcbWHU5VL0q4YgjryZL6ca6oZATTs8kARtZ8spteMDJ8U/P62qxAUCeRSS4bNG0IPn83c2NT8s2zXKJr3XjXYmufVVd1giMPybEGZ9Tz9KjlmKvWuZgqSHhGXSjY/ArFD0cvdf0gsFMojiAhGXJpWuYO6qz9ERs5BcibNGINym9uirQxs+gKBHBuEa77snZ1A==; Original-Received: from [2604:4080:1321:8ab0:bd59:a6b8:f518:ad3f] by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1enoqu-0000tO-Km; Mon, 19 Feb 2018 09:03:44 -0800 In-Reply-To: Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:222900 Archived-At: On 02/19/2018 05:30 AM, Ken Brown wrote: > On 2/17/2018 7:02 PM, Daniel Colascione wrote: >> On Feb 17, 2018 3:38 PM, Ken Brown wrote: >> >>     On 2/15/2018 9:36 PM, Daniel Colascione wrote: >>      > On 02/15/2018 05:56 PM, Ken Brown wrote: >>      >> On 2/15/2018 6:36 PM, Daniel Colascione wrote: >>      >>> On Feb 15, 2018 3:31 PM, Ken Brown wrote: >>     [...] >>      >>>     I just tried to build on 64-bit Cygwin, and the build >> fails as >>      >>> follows: >>      >>> >>      >>>     [...] >>      >>>     Dumping under the name bootstrap-emacs.pdmp >>      >>>     dumping fingerprint: >>      >>> >> 923050a9f611ad7ead76eea704308e4d05f152601a9134cf8d1b5ff3e0e1a986 >>      >>>     Dump complete >>      >>>     Byte counts: header=80 hot=13187392 discardable=119424 >>     cold=9086640 >>      >>>     Reloc counts: hot=919268 discardable=5790 >>      >>>     make -C ../lisp compile-first >>     EMACS="../src/bootstrap-emacs.exe" >>      >>>     make[2]: Entering directory >>      >>>     '/home/kbrown/src/emacs/x86_64-pdumper/lisp' >>      >>>         ELC      ../../pdumper/lisp/emacs-lisp/macroexp.elc >>      >>>         ELC      ../../pdumper/lisp/emacs-lisp/cconv.elc >>      >>>         ELC      ../../pdumper/lisp/emacs-lisp/byte-opt.elc >>      >>>         ELC      ../../pdumper/lisp/emacs-lisp/bytecomp.elc >>      >>>     emacs: could not load dump file >>     "../src/bootstrap-emacs.pdmp": out >>      >>>     of memory >>      >>> >>      >>>     There's probably some obvious explanation, but I don't see >>     it at the >>      >>>     moment. >>      >>> >>      >>> >>      >>> I'm not entirely surprised to see Cygwin fall over here. We >> could >>      >>> just use the Windows memory mapping functions directly, but I'd >>      >>> prefer to stick with the POSIX API if we can make it work. >>      >> >>      >> I agree. >>      >> >>      >>> Any idea where in the mmap sequence we fail? >>      >> >>      >> I haven't looked at the code yet, so I don't understand the >>     question. >>      >> If you give me some guidance, I'll try to investigate. >>      > >>      > Thanks. I think the trouble must be in dump_mmap_contiguous. We >>     report >>      > all errors from this function as "out of memory". >>     dump_mmap_contiguous >>      > takes an array of mapping descriptors (think ELF segments or >>     something) >>      > and maps them all into a single contiguous region of virtual >>     memory. On >>      > POSIX systems, we reserve a chunk of address space with a big >>     PROT_NONE >>      > anonymous mapping, then carve it up into the separate mappings >>     that we >>      > really want. Windows doesn't support atomic mmap replacement, so >>     Cygwin >>      > has to emulate it, and I wouldn't be surprised if something's >> going >>      > wrong in this emulation. >> >>     You're right that the problem is in dump_mmap_contiguous.  I stepped >>     through it in gdb and found the following: >> >>     1. The call to dump_anonymous_allocate at pdumper.c:4432 succeeds. >> >>     2. We enter the loop at pdumper.c:4451 with i = 0.  dump_map_file is >>     called, which calls dump_map_file_posix with protection = >>     DUMP_MEMORY_ACCESS_READWRITE. >> >>     3. This calls mmap at line 4219 with mem_prot = PROT_READ | >> PROT_WRITE >>     and mem_flags = MAP_PRIVATE | MAP_FIXED. >> >>     4. This mmap call fails. >> >>     I don't know if this is a bug in Cygwin's mmap or simply a limitation >>     that has to be worked around.  Should I take this to the Cygwin >> mailing >>     list, or do you have other ideas as to how to proceed? >> >> I'd definitely let the Cygwin people know. If the problem is that >> Cygwin doesn't properly support atomic map replacement, I think we can >> work around the problem by using the same unmap-before-remap approach >> we use for the native Windows code, but keep using the POSIX memory >> functions. I can come up with a patch blind, but you'd be in a better >> position to iterate quickly. > > OK, here's Corinna's answer: > >   https://cygwin.com/ml/cygwin/2018-02/msg00202.html > > It seems like she's suggesting the same unmap-before-remap approach that > you mentioned. Thanks. I've pushed a fix, I think. Try it now.