From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: Re: Should we land Lisp reader optimizations? Date: Wed, 21 Jun 2017 06:07:01 -0400 Message-ID: <8A56FEA9-0BE9-4344-AF29-49EE5405EC08@raeburn.org> References: <83y3snx6b7.fsf@gnu.org> <56EA228B-387F-4983-A91E-97ACFE56F42F@raeburn.org> <83a852wuk0.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1498039634 15590 195.159.176.226 (21 Jun 2017 10:07:14 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 21 Jun 2017 10:07:14 +0000 (UTC) Cc: Eli Zaretskii , emacs-devel@gnu.org To: michael schuldt Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jun 21 12:07:08 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dNcXU-0003hY-Db for ged-emacs-devel@m.gmane.org; Wed, 21 Jun 2017 12:07:08 +0200 Original-Received: from localhost ([::1]:52841 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dNcXZ-0006nI-IC for ged-emacs-devel@m.gmane.org; Wed, 21 Jun 2017 06:07:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:52451) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dNcXT-0006nC-1c for emacs-devel@gnu.org; Wed, 21 Jun 2017 06:07:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dNcXP-0007pL-Ub for emacs-devel@gnu.org; Wed, 21 Jun 2017 06:07:07 -0400 Original-Received: from mail-qt0-x22f.google.com ([2607:f8b0:400d:c0d::22f]:33614) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dNcXP-0007p4-MP for emacs-devel@gnu.org; Wed, 21 Jun 2017 06:07:03 -0400 Original-Received: by mail-qt0-x22f.google.com with SMTP id u12so153272584qth.0 for ; Wed, 21 Jun 2017 03:07:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raeburn-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=P3vWrtR2kdw57uNWrhY6Kps7H2IUbFCCATUfdUlRAL8=; b=BIVYMceteoK6b0lNxFzMVE8IP3bl+xa6n6/TlstwdnUuqBGmW5JLrf6NyADGzBCM17 N9MWBcOZRlAne3+44w968rA4vVpLtKrwPym6j3HSg6KC03peyq3WXkDvonSWKuJyLhoi noVdiEhXwwXUk9afPwZGpzG9nHqmqP0zB3OcalMR5uz8fqvr3DG3//a516xRSidB6Mbn 3deybidLgTB8K+OUEvbVA8BF8s9wwhNfsRljxMojKmi3KH91Swun3jfnQlYJgfmvraLz Ny0nMQPKNX7r51r27/Vd0yzHee8jUQ+WnFSAXeCiw1Vv35r19pgTeu1BRigGW+AR8V+S /o5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=P3vWrtR2kdw57uNWrhY6Kps7H2IUbFCCATUfdUlRAL8=; b=dAxPfDkjEuBCbXnq5OsIED3XQ0y+gHop+c9SnWGGBqZq6hnSpH2/rHDsf1WYmYJS9F j2CPLakfcnbhu4mt8BqseBHDDXzfgKSRTup9TZUcHg2Yzx5uA7yQ/yr1tbryJpBrZrg0 mSyiPMpjSXTAJt3PSJDlu+tYSLuT15j+HY55hEvFHxFtlMbkUlnmhyglN5K80lGM7Kw+ SadHAWKn1kIJqq7azc6EgkUuzehaP5lLEI+Z3IEQmcfPJ/cXHlPafPzlJZ4FAibXqt/1 zZ1PBTCi137uaoVhyrFvI+/4STa5sPSiTK5Yyq0VdRE7/rX3PseDGC/D068KgJ+FUGGR 5o6g== X-Gm-Message-State: AKS2vOxeSEBwxNTcdY8GCFT3qgnhroTA8vvcWKOWTIoJnjwT7sLbAtaF je3N5mdwWbZx1eoP X-Received: by 10.200.47.60 with SMTP id j57mr9724834qta.175.1498039622995; Wed, 21 Jun 2017 03:07:02 -0700 (PDT) Original-Received: from [192.168.23.52] (c-73-253-167-23.hsd1.ma.comcast.net. [73.253.167.23]) by smtp.gmail.com with ESMTPSA id c189sm9812970qkb.7.2017.06.21.03.07.01 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 21 Jun 2017 03:07:02 -0700 (PDT) In-Reply-To: X-Mailer: Apple Mail (2.3124) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400d:c0d::22f X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:215854 Archived-At: On Jun 20, 2017, at 22:50, michael schuldt wrote: > Since the time spent in GC appears so significant, why not disable GC = while reading? >=20 > I have not followed all the previous threads so apologies if this = question is uninformed GC doesn=E2=80=99t happen during reading per se, it happens during = evaluation of expressions and when stopping to wait for user input, but = whether it happens at those points depends on the amount of storage = allocated since the last GC pass. It=E2=80=99s probably possible to = come up with GC heuristics that do better than what we=E2=80=99ve got = now, but there are tradeoffs. The trick is coming up with the right = metrics (memory size? CPU time? delay in interactive response? startup = speed?) and the right use cases to optimize for. It=E2=80=99d be = relatively easy to improve a couple of numbers, at the cost of a worse = experience in other important cases. GC improvement would be a = significant research project of its own, one I=E2=80=99m not solving = this week. :-) But finding simple ways to avoid doing allocations in the first place is = a pretty clear win. Sometimes you get lucky. My test was parsing each Lisp expression in each of 1447 .elc files = (total size over 50MB) and looping over all of that 10 times; that means = we started with something like one GC pass per 16 files processed, on = average, which doesn=E2=80=99t seem so bad. (It would be more frequent = if we were actually evaluating the Lisp expressions that were read. But = I=E2=80=99m only working on the reader code with these changes.) With = the various patches, it=E2=80=99s more like one GC pass per 23 files = scanned. Disabling GC for the duration of reading an entire file probably = wouldn=E2=80=99t make much difference, then. This one-GC-per-16-files = thing is an average, of course; I=E2=80=99d guess the GC probably took = place between reading one expression from a file and reading the next = expression from the same file, but would there be any benefit from = delaying it until we were between files? Raising the gc-cons-threshold value so that GC happens less often could = also be done. But then you=E2=80=99re likely to get faster memory = growth. It=E2=80=99s something that can be considered, but for the = purposes of evaluating changes to the Lisp reader, I figure one less = knob to fiddle with is simplest. In the scratch branch I=E2=80=99ve been working on, with Stefan=E2=80=99s = code to load a saved Lisp environment as one big .elc file, I have for = now raised the gc-cons-threshold value considerably, because the one big = .elc file is big enough to trigger multiple GC passes, and the entire = file has to be read and evaluated before interactive use of the Emacs = process can start. But it=E2=80=99s not a very good =E2=80=9Cfix=E2=80=9D= , as it=E2=80=99ll affect a lot more than startup. Perhaps I should = reset it to its normal value at the end of loading the file, but = that=E2=80=99d likely trigger GC pretty much right away, and I was = trying to avoid triggering any GC delays before getting to the point of = responding to the user=E2=80=99s keyboard input. It=E2=80=99ll have to = be worked out before we can properly evaluate the performance of the = big-elc-file approach=E2=80=A6. Ken=