From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#56682: Fix the long lines font locking related slowdowns Date: Mon, 15 Aug 2022 17:03:44 +0300 Message-ID: <83wnb9hadb.fsf@gnu.org> References: <83h72r16g1.fsf@gnu.org> <640c2e07-98e1-96d6-bb02-19f5f03f637f@yandex.ru> <834jyq29o1.fsf@gnu.org> <92da07bd028e3ede61a6@heytings.org> <47894c57-dd8b-5778-240a-3fa6540e4d37@yandex.ru> <92da07bd02941d5537e9@heytings.org> <5308e3b5-a160-17d7-77ee-b1d00acfa20d@yandex.ru> <92da07bd02a6cc861e1a@heytings.org> <837d3lzv8n.fsf@gnu.org> <2c8d6755-cfe2-6559-3fde-3fa30ffb411e@yandex.ru> <83mtcgy44k.fsf@gnu.org> <83k07jx5jn.fsf@gnu.org> <866e510d-a060-7daa-d002-97861d056fa7@yandex.ru> <1144021660321893@iva5-64778ce1ba26.qloud-c.yandex.net> <12348081660379417@sas2-a098efd00d24.qloud-c.yandex.net> <66bbbb95983414e79637@heytings.org> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36222"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Aug 15 16:07:31 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oNakc-0009F0-4Q for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 15 Aug 2022 16:07:30 +0200 Original-Received: from localhost ([::1]:53160 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oNakb-0002lq-4x for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 15 Aug 2022 10:07:29 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:33622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oNaiE-0000i4-Jk for bug-gnu-emacs@gnu.org; Mon, 15 Aug 2022 10:05:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:53987) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oNaiE-0001Ag-AT for bug-gnu-emacs@gnu.org; Mon, 15 Aug 2022 10:05:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oNaiE-0002jF-5k for bug-gnu-emacs@gnu.org; Mon, 15 Aug 2022 10:05:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 15 Aug 2022 14:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56682 X-GNU-PR-Package: emacs Original-Received: via spool by 56682-submit@debbugs.gnu.org id=B56682.166057224810410 (code B ref 56682); Mon, 15 Aug 2022 14:05:02 +0000 Original-Received: (at 56682) by debbugs.gnu.org; 15 Aug 2022 14:04:08 +0000 Original-Received: from localhost ([127.0.0.1]:43736 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oNahL-0002hq-MR for submit@debbugs.gnu.org; Mon, 15 Aug 2022 10:04:08 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:48664) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oNahJ-0002gw-0p for 56682@debbugs.gnu.org; Mon, 15 Aug 2022 10:04:06 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:39140) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oNahD-00010G-IZ; Mon, 15 Aug 2022 10:03:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=YCc6LClX3K5wx/x01v2zd4vw8E9YTlddG6i8qoZB8oc=; b=rkY8RBYTu5g1 QxhPdYoPX2smk7IiACa3C5bfdoxcTPDYZsFmp2kMzXBSNZsVLRejaX0Owlg9hD8iHI8simHRU+ipO IRGgSvqMPwXJDyL3NPvTJzCPD10UF4FFYAzzN2QrSy9PjvQZiXR9H3EgHKL1NiUuFs1hRRMKY7ITQ xNTR8RMuKxOeQOf/VWX+m02FQCuAmKaT+Qo+0e76i2DM0e0Yb7fNYwJAsgz4nn/TigzzsTkUuJLcg kI2fwRLk+cnK6zvWI9nNsdtSSoMYbVW6vBlOYhRzj57eMkRN2U67Ce90OYzZemLKdAEnB+rtC4k0Z 3XTkexQjd21GL+3Z6Uw0dw==; Original-Received: from [87.69.77.57] (port=4439 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oNahD-0006xr-1p; Mon, 15 Aug 2022 10:03:59 -0400 In-Reply-To: (message from Dmitry Gutov on Sat, 13 Aug 2022 20:20:04 +0300) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:239798 Archived-At: > Date: Sat, 13 Aug 2022 20:20:04 +0300 > Cc: 56682@debbugs.gnu.org, Eli Zaretskii , > monnier@iro.umontreal.ca > From: Dmitry Gutov > > The branch: > > 1) Allows everyone interested to evaluate the performance of > unrestricted font-lock even in large files (single-line or not) and see > how big on a problem the delays caused by syntax-ppss actually are in > their experience. It's an important question. > 2) Figure out the file sizes where syntax-ppss's performance really does > become a problem. That can give us data for better defaults later. > 3) Play around with two easy solutions that we discussed previously: > narrowing during font-lock (one of the values for font-lock-large-files > pretty much matches the current behavior on master), or fontifying only > the first X characters (e.g. 1000000) of the buffer, and skipping on the > rest. > 4) It should be plain to see that implementing additional approaches > should be easy enough. For instance, a hybrid like "fontify the first 1 > MB correctly, and the rest - on best-effort basis". Although the value > '(narrow . 1000000) should provide behavior a very similar behavior > already. Maybe ever a better one: the boundaries are stable. Maybe sure > to try it together with (setq bidi-inhibit-hpa t): the result looks very > fast to me. Thanks. I tested this branch with an unoptimized build and a 416MB file made of 23 copies of dictionary.json, after disabling show-paren-mode. Observations: . With the default value of font-lock-large-files, which allows font-lock to widen as it pleases, the initial M-> in takes 5 min here. That's way too long. On master, this is instantaneous. . Thereafter jumping into various random places inside the file is much faster, but still takes between 1 min and 2.25 min -- again too long, even if I divide by 10 to get an approximation to your faster machine. On master this takes maybe 1 sec, maybe less. . Horizontal and vertical motion commands are as fast (or as slow) as on master, which is not surprising. Bottom line: with the default value of font-lock-large-files, this scales much worse than the master branch -- which again is no surprise. My conclusion is that we do need to prevent uncontrolled widening in fontification-functions, if we want a scalable solution. How small or large should the narrowing be is a separate issue. If we want to allow a more flexible control of that, we could introduce yet another variable exposed to Lisp, so that users and perhaps Lisp programs could tune that, instead of hard-coding the value as we do now on master. I honestly don't see any reason to explore more sophisticated alternatives for where and how to narrow, until and unless major modes will on their side develop capabilities which will make such complications (both in code, but mainly on the user part) justified. If you agree, I'll add on master a variable to control the narrowing when we call fontification-functions.