From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Preview: portable dumper Date: Tue, 29 Nov 2016 10:49:10 -0800 Message-ID: References: <047a67ec-9e29-7e4e-0fb0-24c3e59b5886@dancol.org> <83zikjxt1j.fsf@gnu.org> <727ccd66-3bc3-2a41-7d1d-ef6dae9f0d1e@dancol.org> <7A914757-B1A4-4EEE-9DF0-68EFDDA9A5DB@autodesk.com> <83k2bmxju3.fsf@gnu.org> <837f7mrvq8.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1480445405 7712 195.159.176.226 (29 Nov 2016 18:50:05 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 29 Nov 2016 18:50:05 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux) Cc: John Wiegley , burton.samograd@autodesk.com, rms@gnu.org, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Nov 29 19:50:01 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cBnTZ-0000h4-RK for ged-emacs-devel@m.gmane.org; Tue, 29 Nov 2016 19:49:58 +0100 Original-Received: from localhost ([::1]:38791 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cBnTd-0001b4-MU for ged-emacs-devel@m.gmane.org; Tue, 29 Nov 2016 13:50:01 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:41436) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cBnSy-0001an-EH for emacs-devel@gnu.org; Tue, 29 Nov 2016 13:49:21 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cBnSx-00044N-GS for emacs-devel@gnu.org; Tue, 29 Nov 2016 13:49:20 -0500 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:54450) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cBnSx-00044B-2k; Tue, 29 Nov 2016 13:49:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References:Subject:Cc:To:From; bh=lXg44DW6wc0q5ky/dsmFqsmIikmM9AEvgM97DrGjMWA=; b=rL4PJ6AIS13OZP0YTT0vI/ZhQFXGWkkxziColk/hnw86OTH2ILyWzM+M3uDISO6ubzggC/g3vFmYEJRf+GKz0xmOs+6fpET9BWCNWpQHhwXSKtK4vW9VvYq7zifQZnX2HLH7NTkBfOf+xsGp/wjyGgKQIh99QsdIud1E9eo5iwMpjStE6+sN0d+P3h0JNSlcFvNkNTnEIhHCAiGlAbpV3YvU/xHKTM2dJTTfZGvDlZ1YkpSpg3G3EiYYQi0StArtBJ4LoGIK4KWRCQQq1FriCex14wbscmqJNcQgBmd3OAPKpg7fNE/5NqTeEZA/PQeoA5WcQHir5/NDyQPr+xII5A==; Original-Received: from [2620:0:1008:1101:a803:88c3:331:8b1] (helo=dancol-glaptop0) by dancol.org with esmtpsa (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.84_2) (envelope-from ) id 1cBnSu-0003D9-1m; Tue, 29 Nov 2016 10:49:16 -0800 In-Reply-To: <837f7mrvq8.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 29 Nov 2016 20:23:27 +0200") X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:209750 Archived-At: On Tue, Nov 29 2016, Eli Zaretskii wrote: >> From: John Wiegley >> Cc: rms@gnu.org, dancol@dancol.org, emacs-devel@gnu.org, >> burton.samograd@autodesk.com >> Date: Tue, 29 Nov 2016 10:03:01 -0800 >> >> >>>>> "EZ" == Eli Zaretskii writes: >> >> EZ> However, a dumper that we control by manually telling it what to dum and >> EZ> what not doesn't have to do that. IMO, that's just something that doesn't >> EZ> need to exist in the brave new unexec-free world. (Note that the "one huge >> EZ> .elc file" method has this built-in.) >> >> I'd still like to see research into the "one huge .elc file" method, for the >> obvious benefits you listed. > > Stefan and Ken made some good progress. IMO, we should actively > continue pursuing that goal. > >> As far as I understand, the only difference between the portable >> dumper, and one-huge-.elc, is efficiency. Given infinite resources, >> the .elc method is much to be preferred, since everything we need >> "falls out" from existing design. > > Any memory dumper will always be more efficient than loading .elc. > The challenge is to make the difference smaller than the annoyance > level of the users. Since initial testing already produces load times > below 0.5 sec, I don't see why further optimizations won't deliver > enough speedup to make it fast enough. Because you can't possible get away, in the ELC approach, from: 1) reading every damn byte of the file, and 2) allocating memory for every object you read. Efficiency is always going to be at least linear in the size of the "elc dump" --- super-linear complexity if you want read-circle. A memory dump is linear only in the amount of data you actually access. Plus, there's no serialization or deserialization overhead. A memory dump is also inherently gentler on the underlying system, since we can use many of the pages unmodified (which makes them shareable and trivially swappable), while every damn byte of the Emacs heap after loading ELC files will be process-private dirty storage. I do not think the portable dumper is something that's so complex as to not be worth the performance, especially since any lread implementation that's even close to being acceptably fast is going to be very complex in its own right.