From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: Re: Skipping unexec via a big .elc file Date: Sun, 30 Oct 2016 10:43:31 -0400 Message-ID: <52BDCC33-546C-4F47-A230-00EBC813B038@raeburn.org> References: <83h98nidvd.fsf@gnu.org> <87eg3rvtsf.fsf@users.sourceforge.net> <83k2dihpm9.fsf@gnu.org> <8760p2wzgj.fsf@users.sourceforge.net> <838ttyhhzu.fsf@gnu.org> <871szqwu51.fsf@users.sourceforge.net> <831szqhbc2.fsf@gnu.org> <87d1itt79z.fsf_-_@users.sourceforge.net> <7baa18d4-2b09-caa8-005e-29008a383ad1@cs.ucla.edu> <83mvhwrgd5.fsf@gnu.org> <8539f38f-9a11-44c3-4de7-bb974c96206c@cs.ucla.edu> <8360ojpndr.fsf@gnu.org> <83shrnm0k1.fsf@gnu.org> <075B0922-F07A-4FBA-AE71-027E964A5ED4@raeburn.org> <54AAC13A-CF56-4393-A932-DC6CBBF51259@raeburn.org> <3CC6BB36-1794-4202-8243-132E0345B236@raeburn.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1477838676 1604 195.159.176.226 (30 Oct 2016 14:44:36 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 30 Oct 2016 14:44:36 +0000 (UTC) Cc: Eli Zaretskii , emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Oct 30 15:44:32 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c0rLW-0007Jz-90 for ged-emacs-devel@m.gmane.org; Sun, 30 Oct 2016 15:44:26 +0100 Original-Received: from localhost ([::1]:58764 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c0rLY-0005vM-Tq for ged-emacs-devel@m.gmane.org; Sun, 30 Oct 2016 10:44:28 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44109) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c0rKl-0005vB-Ij for emacs-devel@gnu.org; Sun, 30 Oct 2016 10:43:40 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c0rKh-0002jh-Nh for emacs-devel@gnu.org; Sun, 30 Oct 2016 10:43:39 -0400 Original-Received: from mail-qk0-x229.google.com ([2607:f8b0:400d:c09::229]:32956) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1c0rKh-0002jR-HN for emacs-devel@gnu.org; Sun, 30 Oct 2016 10:43:35 -0400 Original-Received: by mail-qk0-x229.google.com with SMTP id v138so46617259qka.0 for ; Sun, 30 Oct 2016 07:43:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raeburn-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=CkK6HeSuPCcVNFtsnWPhMMcUx/eUkSqFMR3yuo+ul84=; b=hdvos5bAygCriXJi8FA8p89SPboUHx1p/NtnAyrmZHekXmH+qJUa4kXIy/4yzLmtSv 4C3tQJ4bdsKyXNzfFjvk5xDEUutuZF1Rq2XfZ8vP1covC03GmOEubp2cKyKaeh4qHxkU uHuICRj0Mgnh8vmMVZJ3U9DEL+8tXxsAy4b5yxOJd5tfgh0LM2NQ/bjJ2DXUJXb5ew7m bjuu8pSklUiZIgc23nQqOFs3R7Zgu82upgXZfPds+VE30fUallG8znKUt4TfIlJn5Edx Y2y91T0XtgUYawG+12Rxme9DJ1uegu4YXt7UGllwmsV8/omsKH9e7vKFKP2vM4LytOdp AEDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=CkK6HeSuPCcVNFtsnWPhMMcUx/eUkSqFMR3yuo+ul84=; b=ahLRWQBGbsd5rlkdKr5wRYZ+5KxGlpGRtwNhB7yPeO3mXukUwRYsfFwWPU8oBVITiu 8wI7OObFGy/t+BHlXNlf7LIZshDI2YzMMVqtu6opMPQlVkbN4LHWevAEAK9nw+K6p4pA APLPWM7BMNeKYWzbvZA9m1EYxjCXkYyF6VhEJtrm5FLk4hK4brrtn7tELMcxnzkgQPAk N0lekT7L2twS++6MLd3/UjvZirL70Dw4ymT/uEyuA73J446V2/uCn1VSeru3VB4mefb0 wHWohzdrSXV8vUIi6eVG/WzLxhdWkje0ayvZa7K7Ic0UMj86/PExHISaaN/KooVraBrx IE8Q== X-Gm-Message-State: ABUngvd7gX7mUVL8cb1WSxGwBxcWLvsdzp2PmDvfuKI+PRUi5e7WFLKNtcm45U4lJN7iGw== X-Received: by 10.55.79.17 with SMTP id d17mr10829769qkb.110.1477838614388; Sun, 30 Oct 2016 07:43:34 -0700 (PDT) Original-Received: from [192.168.23.52] (c-50-138-183-136.hsd1.ma.comcast.net. [50.138.183.136]) by smtp.gmail.com with ESMTPSA id 16sm11083342qtf.36.2016.10.30.07.43.33 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 30 Oct 2016 07:43:33 -0700 (PDT) In-Reply-To: <3CC6BB36-1794-4202-8243-132E0345B236@raeburn.org> X-Mailer: Apple Mail (2.3124) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400d:c09::229 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208987 Archived-At: I wrote: > Patched substitute_object_recurse after the check for the subtree = matching the placeholder, so that if the subtree passed was a symbol or = number, it would simply be returned without consulting seen_list or = read_objects. Run time is now 0.7s; Fassq is a bit over 50% of that, = and Frassq about 17%, and _IO_getc around 11%. I think it should be = safe to short-circuit it for some other types as well. >=20 > I had my getc_unlocked change sitting around so I pulled that in. Run = time is now 0.6s, with Fassq at 57% and Frassq at 18%. >=20 > Next on the profiling chart is oblookup, but it=E2=80=99s only at 4% = so I=E2=80=99m going to ignore OBARRAY_SIZE for now. However, = OBARRAY_SIZE could affect the order of atoms in processing, which could = drastically rearrange the ordering of the data structures in dumped.elc. >=20 > I think the next step is to look at replacing read_objects, probably = with a pair of hash tables, but it=E2=80=99s getting a bit late for = trying that tonight. I switched over to a pair of hash tables and the run time is just under = 0.2s on my test machine now. Profiling reports are now topped by read1, = readchar, and readbyte_from_file (now including the expanded = getc_unlocked calls), accounting for about 30% of the CPU time between = them. The hash functions and substitute_object_recurse are not taking a = significant amount of time. I took a look at the types of shared data in one of the generated = dumped.elc files I got; almost 2700 were strings (all without text = properties), almost 1900 were cons cells, and the rest numbered under = 300. So I=E2=80=99m not sure special-casing other types besides = Lisp_Cons in read1 will gain us much. It took me a while to sort through the lookups being done during and = after parsing of an object and how the checks for circular objects work, = but I think I=E2=80=99ve got it working. I=E2=80=99ve pushed a scratch = branch over with the changes if you=E2=80=99d like to try them, though I = think I botched the git push syntax when trying to create = =E2=80=9Cscratch/raeburn/startup=E2=80=9D somehow, so I created = =E2=80=9Cscratch/raeburn-startup=E2=80=9D=E2=80=A6 or possibly I=E2=80=99v= e created both? I saw an email notification go out for both, but I only = see the latter in the repository browser interface=E2=80=A6=20 Ken=