From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Ken Brown Newsgroups: gmane.emacs.devel Subject: Re: Preview: portable dumper Date: Mon, 19 Feb 2018 08:30:46 -0500 Message-ID: References: <797cf77a-337e-459a-a82b-28d2c63418a7@email.android.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1519046965 11912 195.159.176.226 (19 Feb 2018 13:29:25 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 19 Feb 2018 13:29:25 +0000 (UTC) User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 Cc: Eli Zaretskii , emacs-devel@gnu.org To: Daniel Colascione Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Feb 19 14:29:20 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1enlVD-00024a-Iw for ged-emacs-devel@m.gmane.org; Mon, 19 Feb 2018 14:29:07 +0100 Original-Received: from localhost ([::1]:50374 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1enlXF-0005sH-Lo for ged-emacs-devel@m.gmane.org; Mon, 19 Feb 2018 08:31:13 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51483) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1enlX3-0005qC-4M for emacs-devel@gnu.org; Mon, 19 Feb 2018 08:31:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1enlWz-0002Ip-4F for emacs-devel@gnu.org; Mon, 19 Feb 2018 08:31:01 -0500 Original-Received: from limerock04.mail.cornell.edu ([128.84.13.244]:33460) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1enlWy-0002IZ-VP; Mon, 19 Feb 2018 08:30:57 -0500 X-CornellRouted: This message has been Routed already. Original-Received: from authusersmtp.mail.cornell.edu (granite3.serverfarm.cornell.edu [10.16.197.8]) by limerock04.mail.cornell.edu (8.14.4/8.14.4_cu) with ESMTP id w1JDUmOj001894; Mon, 19 Feb 2018 08:30:48 -0500 Original-Received: from [192.168.0.15] (mta-68-175-129-7.twcny.rr.com [68.175.129.7] (may be forged)) (authenticated bits=0) by authusersmtp.mail.cornell.edu (8.14.4/8.12.10) with ESMTP id w1JDUksT017826 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 19 Feb 2018 08:30:47 -0500 In-Reply-To: <797cf77a-337e-459a-a82b-28d2c63418a7@email.android.com> Content-Language: en-US X-PMX-Cornell-Gauge: Gauge=XXXXX X-PMX-CORNELL-AUTH-RESULTS: dkim-out=none; X-MIME-Autoconverted: from 8bit to quoted-printable by limerock04.mail.cornell.edu id w1JDUmOj001894 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 128.84.13.244 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:222896 Archived-At: On 2/17/2018 7:02 PM, Daniel Colascione wrote: > On Feb 17, 2018 3:38 PM, Ken Brown wrote: >=20 > On 2/15/2018 9:36 PM, Daniel Colascione wrote: > > On 02/15/2018 05:56 PM, Ken Brown wrote: > >> On 2/15/2018 6:36 PM, Daniel Colascione wrote: > >>> On Feb 15, 2018 3:31 PM, Ken Brown wrote: > [...] > >>> =C2=A0=C2=A0=C2=A0 I just tried to build on 64-bit Cygwin, and= the build fails as > >>> follows: > >>> > >>> =C2=A0=C2=A0=C2=A0 [...] > >>> =C2=A0=C2=A0=C2=A0 Dumping under the name bootstrap-emacs.pdmp > >>> =C2=A0=C2=A0=C2=A0 dumping fingerprint: > >>> =20 > 923050a9f611ad7ead76eea704308e4d05f152601a9134cf8d1b5ff3e0e1a986 > >>> =C2=A0=C2=A0=C2=A0 Dump complete > >>> =C2=A0=C2=A0=C2=A0 Byte counts: header=3D80 hot=3D13187392 dis= cardable=3D119424 > cold=3D9086640 > >>> =C2=A0=C2=A0=C2=A0 Reloc counts: hot=3D919268 discardable=3D57= 90 > >>> =C2=A0=C2=A0=C2=A0 make -C ../lisp compile-first > EMACS=3D"../src/bootstrap-emacs.exe" > >>> =C2=A0=C2=A0=C2=A0 make[2]: Entering directory > >>> =C2=A0=C2=A0=C2=A0 '/home/kbrown/src/emacs/x86_64-pdumper/lisp= ' > >>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 ELC=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 ../../pdumper/lisp/emacs-lisp/macroexp.elc > >>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 ELC=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 ../../pdumper/lisp/emacs-lisp/cconv.elc > >>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 ELC=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 ../../pdumper/lisp/emacs-lisp/byte-opt.elc > >>> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0 ELC=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 ../../pdumper/lisp/emacs-lisp/bytecomp.elc > >>> =C2=A0=C2=A0=C2=A0 emacs: could not load dump file > "../src/bootstrap-emacs.pdmp": out > >>> =C2=A0=C2=A0=C2=A0 of memory > >>> > >>> =C2=A0=C2=A0=C2=A0 There's probably some obvious explanation, = but I don't see > it at the > >>> =C2=A0=C2=A0=C2=A0 moment. > >>> > >>> > >>> I'm not entirely surprised to see Cygwin fall over here. We co= uld > >>> just use the Windows memory mapping functions directly, but I'= d > >>> prefer to stick with the POSIX API if we can make it work. > >> > >> I agree. > >> > >>> Any idea where in the mmap sequence we fail? > >> > >> I haven't looked at the code yet, so I don't understand the > question. > >> If you give me some guidance, I'll try to investigate. > > > > Thanks. I think the trouble must be in dump_mmap_contiguous. We > report > > all errors from this function as "out of memory". > dump_mmap_contiguous > > takes an array of mapping descriptors (think ELF segments or > something) > > and maps them all into a single contiguous region of virtual > memory. On > > POSIX systems, we reserve a chunk of address space with a big > PROT_NONE > > anonymous mapping, then carve it up into the separate mappings > that we > > really want. Windows doesn't support atomic mmap replacement, so > Cygwin > > has to emulate it, and I wouldn't be surprised if something's go= ing > > wrong in this emulation. >=20 > You're right that the problem is in dump_mmap_contiguous.=C2=A0 I s= tepped > through it in gdb and found the following: >=20 > 1. The call to dump_anonymous_allocate at pdumper.c:4432 succeeds. >=20 > 2. We enter the loop at pdumper.c:4451 with i =3D 0.=C2=A0 dump_map= _file is > called, which calls dump_map_file_posix with protection =3D > DUMP_MEMORY_ACCESS_READWRITE. >=20 > 3. This calls mmap at line 4219 with mem_prot =3D PROT_READ | PROT_= WRITE > and mem_flags =3D MAP_PRIVATE | MAP_FIXED. >=20 > 4. This mmap call fails. >=20 > I don't know if this is a bug in Cygwin's mmap or simply a limitati= on > that has to be worked around.=C2=A0 Should I take this to the Cygwi= n mailing > list, or do you have other ideas as to how to proceed? >=20 > I'd definitely let the Cygwin people know. If the problem is that Cygwi= n=20 > doesn't properly support atomic map replacement, I think we can work=20 > around the problem by using the same unmap-before-remap approach we use= =20 > for the native Windows code, but keep using the POSIX memory functions.= =20 > I can come up with a patch blind, but you'd be in a better position to=20 > iterate quickly. OK, here's Corinna's answer: https://cygwin.com/ml/cygwin/2018-02/msg00202.html It seems like she's suggesting the same unmap-before-remap approach that=20 you mentioned. Ken