From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: Re: Skipping unexec via a big .elc file Date: Thu, 27 Oct 2016 04:51:24 -0400 Message-ID: <3CC6BB36-1794-4202-8243-132E0345B236@raeburn.org> References: <83h98nidvd.fsf@gnu.org> <87eg3rvtsf.fsf@users.sourceforge.net> <83k2dihpm9.fsf@gnu.org> <8760p2wzgj.fsf@users.sourceforge.net> <838ttyhhzu.fsf@gnu.org> <871szqwu51.fsf@users.sourceforge.net> <831szqhbc2.fsf@gnu.org> <87d1itt79z.fsf_-_@users.sourceforge.net> <7baa18d4-2b09-caa8-005e-29008a383ad1@cs.ucla.edu> <83mvhwrgd5.fsf@gnu.org> <8539f38f-9a11-44c3-4de7-bb974c96206c@cs.ucla.edu> <8360ojpndr.fsf@gnu.org> <83shrnm0k1.fsf@gnu.org> <075B0922-F07A-4FBA-AE71-027E964A5ED4@raeburn.org> <54AAC13A-CF56-4393-A932-DC6CBBF51259@raeburn.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1477558359 24028 195.159.176.226 (27 Oct 2016 08:52:39 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 27 Oct 2016 08:52:39 +0000 (UTC) Cc: Eli Zaretskii , emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Oct 27 10:52:34 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bzgQG-0004rM-PV for ged-emacs-devel@m.gmane.org; Thu, 27 Oct 2016 10:52:29 +0200 Original-Received: from localhost ([::1]:39749 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzgQJ-0003nS-8g for ged-emacs-devel@m.gmane.org; Thu, 27 Oct 2016 04:52:31 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40211) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzgPN-0003nK-Rh for emacs-devel@gnu.org; Thu, 27 Oct 2016 04:51:38 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bzgPJ-0001Dp-0I for emacs-devel@gnu.org; Thu, 27 Oct 2016 04:51:33 -0400 Original-Received: from mail-qt0-x22a.google.com ([2607:f8b0:400d:c0d::22a]:36823) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1bzgPI-0001DN-Qc for emacs-devel@gnu.org; Thu, 27 Oct 2016 04:51:28 -0400 Original-Received: by mail-qt0-x22a.google.com with SMTP id 12so17763473qtm.3 for ; Thu, 27 Oct 2016 01:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raeburn-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=g9GxZNbQbpRShtlnadOcpmwF7NkUFh8SXMEZrehXiSg=; b=Zvxi2sOgU8CH6Qr9Vg4N+606lCpmHSEutZLEMnLm6cnjlFHRB1erN8EA1PF530zfDa NmWGVYclE0jv8LzXzQjc+FrLIe3TpXDTBPvVSBnFQdqY/RveuDzQvyJZo6OrqDnri3Uf SUnB/cLzH751CIbjLFbBjTdpQ4FhvGlE4NdRBnCw2/XPFGzIvAmG023ykW0SQnVEosQ4 mfqi+NUqyQR8DXo3WYeYiR5TMNjQVx9Nf45j4psyAOLRYvAvHzK2u+/4FK1OM2JIT/j4 SagfghVfPu0/Wa4UD7umEzP0kfQMwVOXGf5kteCoxj/59moCszB0Nvh/oJ6LJ90Z/Jh6 4j/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=g9GxZNbQbpRShtlnadOcpmwF7NkUFh8SXMEZrehXiSg=; b=aQWXnaFPe9mZVKD3I2dZfWVk7Th8hllJcciJiGjhhBorrKHOLL7oRzZufSK0Znid8Q 0CjKIX8AuNOxVYjGqS4bwSxM4aGeVdQb5ezUg3vSLUSfTO7UFSV02xKzOKribZjstdxg iasfV9uIIgVb7+CclG3J07y4YcjIFlk7Bua2j1qRiaNIx8NvU8R2U9QS2RADHH4nX605 da0WcwXsPXcNyhS/YwpkEsVrjzpy8SLKlLqRu68Tnq6ZS5No4y8TfnvxvlYvvleWpCuU V3sQbPhYWIKCDQ5haFKvEtT0HzJJFVpaPvtyzFHHcu1d6VgW3RLRlgSVF7bf/QRNcNL5 500w== X-Gm-Message-State: ABUngvehkcESrrVl9Ixgu4dqqqMpM5zqDxt3oBlbgJXGCCfhjF3iMCd2Fbo4uzd7u4LMuA== X-Received: by 10.237.34.149 with SMTP id p21mr5571598qtc.84.1477558287615; Thu, 27 Oct 2016 01:51:27 -0700 (PDT) Original-Received: from [192.168.23.52] (c-50-138-183-136.hsd1.ma.comcast.net. [50.138.183.136]) by smtp.gmail.com with ESMTPSA id l73sm3113572qke.26.2016.10.27.01.51.26 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Oct 2016 01:51:26 -0700 (PDT) In-Reply-To: X-Mailer: Apple Mail (2.3124) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400d:c0d::22a X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208865 Archived-At: > On Oct 25, 2016, at 09:48, Stefan Monnier = wrote: >=20 >>>> Did you check whether actually byte compiling the written file made >>>> a difference? >>> dumped.elc has no code to compile. >> It has a lot of fset and setplist calls which can be compiled, = especially if >> you reorder things such that they=E2=80=99re not mixed up with the = defvar calls that >> don=E2=80=99t compile. >=20 > "A lot of" is relative: the time to read them compared to an = equivalent > byte-code version should be negligeable, and their execution time = should > be even more negligeable. >=20 >> The generated .elc output is about 25% larger. >=20 > That's not because of byte-compilation per-se. It's because the > byte-compiler uses `print-circle' but only within each top-level = entity, > so you lose sharing between functions and between variables. >=20 > IOW you can get the exact same 25% larger file by printing each > fset/defvar/setplist separately (instead of printing them as one big > `progn`). And you can trick the byte-compiler to preserve this = sharing > by replacing the leading `progn` (which the byte-compiler removes) = into > a (let () ...), tho maybe you'll need to really add some dummy binding > in that `let` to make sure the byte-compiler doesn't end up removing = it. Ah, yes=E2=80=A6 =E2=80=9C(let () =E2=80=A6)=E2=80=9D was enough with no = bindings. Now the compiled file, which now contains only one big = byte-code invocation, is still larger than the original dumped file, = though not as much, and from a couple of spot checks it looks like the = data sharing is indeed preserved. It also takes longer to load. Oh = well. > Ideally, we could get rid of substitute_object_in_subtree entirely. > E.g. the patch below skips it for the case of "#n=3D(...)", and by = peeping > ahead to decide the type of placeholder we build, we should be able to > get rid of it in all cases. I would think not for types using flexible array members, since we may = not know the allocation size until we=E2=80=99ve seen the end of the = object. In poking around with gdb, most of the invocations of = substitute_object_in_subtree I looked at got a subtree of nil. It = appears to me that if the =E2=80=9Csubtree=E2=80=9D passed isn=E2=80=99t = the placeholder and isn=E2=80=99t one of the types we process = recursively, then we will never do any substitution, right? So the = checking of seen_list and read_objects isn=E2=80=99t relevant. I started my tests over with an updated source tree from upstream and = put in your loadup.el change. Running =E2=80=9Ctime emacs -batch -l = dumped.elc=E2=80=9D took 3.5s; according to =E2=80=9Cperf = record=E2=80=9D/=E2=80=9Cperf report=E2=80=9D, Frassq took about 85% of = the CPU time, and Fassq took about 9%. Added your lread.c patch; run time is about 1.8s, 70% in Frassq and = almost 20% in Fassq. Patched substitute_object_recurse after the check for the subtree = matching the placeholder, so that if the subtree passed was a symbol or = number, it would simply be returned without consulting seen_list or = read_objects. Run time is now 0.7s; Fassq is a bit over 50% of that, = and Frassq about 17%, and _IO_getc around 11%. I think it should be = safe to short-circuit it for some other types as well. I had my getc_unlocked change sitting around so I pulled that in. Run = time is now 0.6s, with Fassq at 57% and Frassq at 18%. Next on the profiling chart is oblookup, but it=E2=80=99s only at 4% so = I=E2=80=99m going to ignore OBARRAY_SIZE for now. However, OBARRAY_SIZE = could affect the order of atoms in processing, which could drastically = rearrange the ordering of the data structures in dumped.elc. I think the next step is to look at replacing read_objects, probably = with a pair of hash tables, but it=E2=80=99s getting a bit late for = trying that tonight. Ken=