From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: "Daniel Colascione" Newsgroups: gmane.emacs.devel Subject: Re: pdumper's performance Date: Wed, 29 Aug 2018 22:19:27 -0700 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1535607039 17100 195.159.176.226 (30 Aug 2018 05:30:39 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 30 Aug 2018 05:30:39 +0000 (UTC) User-Agent: SquirrelMail/1.4.23 [SVN] Cc: Daniel Colascione , emacs-devel@gnu.org To: "Stefan Monnier" Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Aug 30 07:30:35 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fvFXN-0004IK-Bm for ged-emacs-devel@m.gmane.org; Thu, 30 Aug 2018 07:30:33 +0200 Original-Received: from localhost ([::1]:47137 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fvFZT-0004n4-Bf for ged-emacs-devel@m.gmane.org; Thu, 30 Aug 2018 01:32:43 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:34108) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fvFYg-0004j4-6d for emacs-devel@gnu.org; Thu, 30 Aug 2018 01:31:55 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fvFMh-0007tj-7B for emacs-devel@gnu.org; Thu, 30 Aug 2018 01:19:32 -0400 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:46444) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fvFMg-0007tF-Tm for emacs-devel@gnu.org; Thu, 30 Aug 2018 01:19:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:To:From:Subject:Date:References:In-Reply-To:Message-ID; bh=6u9Y3sFjKR+oCs4K72hB4cBRbBRlzwSff5ovtUwi8lk=; b=kiozc4MA8b/T97DOJ80gUdJF/JRuBC2lCZMc9Xj/OytJFNss27J8Dp4jERVi1lsNoSR6cx3LSCc2AkPAuMbKHeMnjG5O2sXdhc72gbcsMYu63R/B3PHEcNLMYeHSxNZQn62lEq+xgJLY3tAwfrWYDXj95h+l7Bg1gC6+1wosyUzPXjGt+LghM6DRwE5ShYWh2sbjroNaYFd6lE8NkQ6r1T1iSg9gDHu81EQ2U96tmr2F/CVKh4E6+PYdFK565oNwSEiXrpJ9ONSGnPwn2mIHoCCh7TCezLikPv6qvd0EFxfA5muOdYdlaBkaamW1Rf2dk9emJMLvF8azgofEICKrKA==; Original-Received: from localhost ([127.0.0.1] helo=dancol.org) by dancol.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fvFMd-0002r2-FZ; Wed, 29 Aug 2018 22:19:27 -0700 Original-Received: from 127.0.0.1 (SquirrelMail authenticated user dancol) by dancol.org with HTTP; Wed, 29 Aug 2018 22:19:27 -0700 In-Reply-To: X-Priority: 3 (Normal) Importance: Normal X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:229088 Archived-At: > Thanks Daniel for your prompt response. I have some further questions, > tho. > >> You can see for yourself whether there's an impact. Compile an Emacs >> with >> support for both pdumper and unexec, dump it with unexec, and compare >> its >> GC performance to Emacs built without support for pdumper and also >> dumped >> with unexec. > > I hoping to save myself the time ;-) > [ BTW, part of the reason for those questions is that I'm writing an > article about the history of Elisp, and I'd like to understand how > your code works so I can say something intelligent about it. > Oh and there's not much time left before the deadline. Cool. > Another part of course, is that I'd like to see this feature land > on master. ] Me too. ;-) > >> As I recall, the difference is minimal. > > Do you recall the tests you used and the ballpark of the difference? Exactly the above. IIRC, the difference amounted to a millisecond or two on an emacs -Q startup plus an immediate (garbage-collect) --- but that's without the no-relocation optimization below. >>> Also I don't quite understand why this is needed: IIUC the markbits of >>> pdump'd objects are stored elsewhere, but I don't understand why that >>> needs to be the case. >> Because we don't store dumped objects in blocks and so the calculations >> of >> the normal locations of their mark bits would be wrong. > > Hmm... OK that could explain it for conses and floats where we keep the > markbits separately from the objects in bitmaps alongside those blocs, > but you also have those _marked_p and set__marked functions for > all other types of objects where the markbit is normally stored within > the object itself (i.e. it doesn't matter whether they're in blocks or > not). > > Why did you choose to use a completely different layout for the objects > loaded from the dump? The objects themselves have the same layout that they do in the normal heap. (The layout of a cons cell is unchanged, for example.) Dumping objects individually instead of in blocks both simplifies the implementation and allows for a more compact dump, as you point out below. > I naively thought your code would take > cons_blocks, symbol_blocks, ... and write those blocks as-is so objects > keep the same layout, and things like mark_maybe_object don't need to be > changed at all. I understand this would end up writing larger dumps > (since they would include some free objects), but I'd have expected it > would lead to simpler code and a smaller patch. If we keep the mark bits out of the objects, we can avoid modifying the object pages just for GC. In the non-PIC case, in which in principle we don't have to relocate the dump, that means that the pages in the dump stay clean and file-backed, not dirty, COWed, and pagefile-backed as they would if we banged on them just for the GC. That's an efficiency win. For a future more-efficient GC, contiguous object storage with external mark bits is probably the way to go for the entire heap.