From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Time to drop the pre-dump phase in the build? Date: Fri, 10 Jan 2014 21:30:13 -0800 Message-ID: <52D0D6E5.9060507@dancol.org> References: <20140110191530.5772E38019B@snark.thyrsus.com> <52D071EC.4090607@dancol.org> <52D08B37.5090505@dancol.org> <52D0BC7C.2000700@dancol.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1389418237 8988 80.91.229.3 (11 Jan 2014 05:30:37 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 11 Jan 2014 05:30:37 +0000 (UTC) Cc: "Eric S. Raymond" , stephen@xemacs.org, Emacs developers To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Jan 11 06:30:44 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1W1r9f-00014k-Ii for ged-emacs-devel@m.gmane.org; Sat, 11 Jan 2014 06:30:43 +0100 Original-Received: from localhost ([::1]:60233 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W1r9e-0004lj-KQ for ged-emacs-devel@m.gmane.org; Sat, 11 Jan 2014 00:30:42 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:54265) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W1r9W-0004lc-7L for emacs-devel@gnu.org; Sat, 11 Jan 2014 00:30:40 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1W1r9P-0007dk-Tv for emacs-devel@gnu.org; Sat, 11 Jan 2014 00:30:34 -0500 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:43761) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W1r9P-0007da-IL for emacs-devel@gnu.org; Sat, 11 Jan 2014 00:30:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:References:Subject:CC:To:MIME-Version:From:Date:Message-ID; bh=COTsCs2a1pf7dkPHqWtRBjQmiwK3JgN7VJnt/scKTE8=; b=Y+1NSsyKhUoAooxlkg9PnC9usuVxPbu7eGI7E+2fYZNEmEAQCHW87LOTW2XNi53wvPaxiTVpnY3r+JSq23HlsO8d1/Pzyl9XFK1KM31rAK61jpIS/VojhvrZazrBLkgZ7AK3QJPHyJYtMWVG+VTmQl0CE/Q6pq+SvNi2GPNvO4KvYuqqxxZ5yP4cyrVdQYbjjRUVODNvgfHB49D3cgWLFsGJB3Rpf+T9zDLeBEDYzDhztXVyp9JVnn9aOiYpzxx2NVeo04TDRST6mkGbMPiql64mYG2aQUmMKKmmaz1h649n460qtswBnFiGXUagY2cM/ijZnsdc2iaPygXemnHuxA==; Original-Received: from [12.130.126.124] (helo=[172.19.131.149]) by dancol.org with esmtpsa (TLS1.0:DHE_RSA_CAMELLIA_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1W1r9M-0001bz-Kn; Fri, 10 Jan 2014 21:30:25 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:168063 Archived-At: On 01/10/2014 09:13 PM, Stefan Monnier wrote: >> Another possibility is to just allocate enough space in the emacs image >> itself in BSS, then replace that mapping with a view of the dump file. > > Indeed, that should work, assuming you can mmap into existing space. On POSIX-y systems, you can just mmap on top of the existing section. On Windows, you have to unmap first, but I think it could be made to work. > But not nearly as bad: the main dump problem we have is with generating > the `emacs' executable, whereas here we'd only need to generate the > "swap file" which is later loaded into the same executable. > Should still be a lot more portable. Do you mean building emacs with a large blob of zero in .data, using it as a heap, and replacing the contents of that section (without modifying the executable image structure) to actually "dump" emacs? >> By the way: is it me, or are we dirtying far too much of the current emacs >> image? On my Emacs, we're dirtying (and COWing) 8MB; if I make >> Fgarbage_collect a no-op, that drops to 4MB. > > For sure, GC will dirty up pretty much all pages that hold Lisp objects > (except for those in the purespace), because of the need to set/reset > the `mark' bit. I was thinking about this problem. What if we were to just treat all image-backed objects as already marked if they're in pages that are unmodified? (We can perform this test very cheaply, at least on */Linux and Windows.) Then we wouldn't mark them during GC, and we additionally don't demand-page objects just for GC. The problem we create is that we might have modified image-backed objects reachable only from unmodified image-backed objects, and these modified objects might point to heap-allocated objects that we really should mark. So what if we walk the per-type allocation lists during the *mark* phase and treat all in-image objects on modified pages as individual roots? This way, we eventually mark all heap-allocated objects. (Let's assume that no image-backed unmodified object can directly point to a heap-allocated object.) This way, we can avoid touching most dumped data structures during GC. We might modify them for other reasons, though, like setting symbol value cells --- but if my quick and dirty GC test worked correctly, we should still save quite a bit on commit charge without worrying about these cases.