From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#56682: Fix the long lines font locking related slowdowns Date: Fri, 5 Aug 2022 04:39:46 +0300 Message-ID: References: <837d46mjen.fsf@gnu.org> <8335esjppt.fsf@gnu.org> <837d43j198.fsf@gnu.org> <83y1wjhkkh.fsf@gnu.org> <83wnc3hkdg.fsf@gnu.org> <49df74e5-e16a-a532-98d1-66c6ff1eb6c6@yandex.ru> <83pmhuft5a.fsf@gnu.org> <05388e8d8836c2e7ef3e@heytings.org> <136c4fe0fcb9ce5181cb@heytings.org> <3d639ea12689d767ba2a@heytings.org> <03dce9be-5c51-94d7-a32a-52ab7f57dde2@yandex.ru> <83r11w2m0i.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32019"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Cc: gerd.moellmann@gmail.com, 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Aug 05 03:40:30 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oJmKD-0008F1-Vk for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 05 Aug 2022 03:40:30 +0200 Original-Received: from localhost ([::1]:49158 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oJmKD-0001a6-1u for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 04 Aug 2022 21:40:29 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:40514) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oJmJm-0001Ym-DW for bug-gnu-emacs@gnu.org; Thu, 04 Aug 2022 21:40:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:37466) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oJmJm-0000qS-4b for bug-gnu-emacs@gnu.org; Thu, 04 Aug 2022 21:40:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oJmJl-0005GE-V9 for bug-gnu-emacs@gnu.org; Thu, 04 Aug 2022 21:40:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 05 Aug 2022 01:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56682 X-GNU-PR-Package: emacs Original-Received: via spool by 56682-submit@debbugs.gnu.org id=B56682.165966360120212 (code B ref 56682); Fri, 05 Aug 2022 01:40:01 +0000 Original-Received: (at 56682) by debbugs.gnu.org; 5 Aug 2022 01:40:01 +0000 Original-Received: from localhost ([127.0.0.1]:55448 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oJmJk-0005Fv-JA for submit@debbugs.gnu.org; Thu, 04 Aug 2022 21:40:01 -0400 Original-Received: from mail-wm1-f51.google.com ([209.85.128.51]:37543) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oJmJe-0005FZ-S6 for 56682@debbugs.gnu.org; Thu, 04 Aug 2022 21:39:59 -0400 Original-Received: by mail-wm1-f51.google.com with SMTP id c187-20020a1c35c4000000b003a30d88fe8eso3302953wma.2 for <56682@debbugs.gnu.org>; Thu, 04 Aug 2022 18:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=hrjwmVBtqOkCjgJ+g9Yw1oUHQk7/JxcFdzQ290qTB/A=; b=SYtxVKwMPL7f9jF14zQ7jsEY2cXghR5ssh+tY+rNJtveWA8coPz9hl1SsouC9QpCAq rYU3CdQTOfLKJOTwZe+oEGow9NKmZbwzxh+oH3QwgJ699H/EpAlgtLUTe3LNGw4Y2caD fcXeYhIirCn9uE/NPRLTh5g9iVUJmIbrEcRtIzdvmArov779MBj4ZWF5J+W7EDTCFW8z cxgnnqrf22OgPpxNH+ebKGLuhgezdPJ3Lw64nRoGHXe8Kx6XxauUxdqpGFS942JUJx2n B8/99ttaZf2U0XjJUY9M3FalMV4fa+MTejNoUh7dwAcCEDwBbdYraz/uXpaVBEWhZCmu WFMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:message-id:date:mime-version:user-agent :subject:content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=hrjwmVBtqOkCjgJ+g9Yw1oUHQk7/JxcFdzQ290qTB/A=; b=ps92mKyMFDJm8DicfgFIIbOyKuoEpKX+i9eFeQlse+llXkgjL+K88Cm6n0B16IkSeU NWbixZih94+h1Qt6WiKSB4WL9ZbpXQAi60UR5jIv7LVUWn5HC+fe/uFj7b1jCtGwz+NX zJXMk28Xo5tozxWk+H4MEAfNSSh7xuPDFml+3pgnx/Age+CpxktBY1yEHwByFKb2k6qF ncZjPRN7/tHt2FM+zubzNNW+EJQTN2V4/7Q44p1q/HpQnFl61i8ZLrrY+af5U+IdfPP4 BIVqal+tCCBBLOq2l52iZjsruzQkL8CT+5c/rZ6ez7PCas6s4SiBzX3CLfnXmX16fN3F iDhg== X-Gm-Message-State: ACgBeo1SR9D1RC4UBdhKXDzjDsQZLh5i4FLAiVFHpzH5U+rtHS6QPY7b RfzrYlOrhEOcYuT40LIDxPI= X-Google-Smtp-Source: AA6agR5IJMqLiYWi7oWNmIWpgIInNj6k2GzTWPg12mKVk2/0QFPGidfmhnLU4Vpb3yq9ItgE5tYCaQ== X-Received: by 2002:a05:600c:3641:b0:3a2:df38:7ec8 with SMTP id y1-20020a05600c364100b003a2df387ec8mr3001906wmq.34.1659663588770; Thu, 04 Aug 2022 18:39:48 -0700 (PDT) Original-Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id j6-20020a05600c190600b003a31b79dc0esm21752449wmq.1.2022.08.04.18.39.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 04 Aug 2022 18:39:48 -0700 (PDT) Content-Language: en-US In-Reply-To: <83r11w2m0i.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:238808 Archived-At: On 04.08.2022 16:09, Eli Zaretskii wrote: >> That's the scenario I described, and that's my point: this file's >> display is sluggish. Even though font-lock has already finished its >> work. And it didn't have to spend any significant time in syntax-ppss. >> >> So there is a particular performance problem with the display of >> fontified buffers which I'd really like your help in fixing. > > Maybe it is in display, and maybe it isn't. Do you have any evidence > that the sluggish response is due to redisplay? C-n, for example, is > mostly not redisplay, but Lisp code in simple.el and occasional calls > to vertical-motion. I come to that conclusion by observing said sluggish movement in a buffer that is fully fontified. And yet the delays after pressing 'C-n' or 'C-p' or 'C-n' seemed noticeable and very similar to delays on PgUp/PgDown. Are these delays in fontification-functions? That seems unlikely given the buffer is fontified already in full, and none of the commands are causing modifications (which would force syntax-ppss cache and fontifications to be recalculated). If the delays are in font-lock anyway, that might be in the code which checks that the area between window-start and window-end is fontified. I don't see why that code has to be slow (long lines or not), so it should be fixable easily enough. Unless 'get-text-property' or 'next-single-property-change' can exhibit pathologic performance in the presence of long lines, of course. But that doesn't show up in my testing. > But even if the slow response is due to redisplay, we just have > another cause that we need to investigate and try fixing. It seems to me that that cause actually has larger impact than font-lock, because it does show itself in a moderately-sized (88K) buffer, where font-lock doesn't feel like a problem. It stands to reason that the same "cause" might have a proportionally bigger impact in large buffers as well, and only after we remove it (alone), then we can evaluate how font-lock itself affects user experience, and how much of its correctness (and for buffers of which size) we want to sacrifice. > It says > nothing about the measures we've already taken on master. They > definitely make even this case faster, and with an unoptimized build I > can now reasonably edit this file, something I couldn't do before. If my guess is right, the fix on master whammied all over the redisplay with narrowing, both fixing the "cause" and restricting font-lock to the same narrowed region. The latter part might be unnecessary in the usual case (we might still decide to do that later for much larger buffers, but that should be decided by a separate threshold variable). >> Fixing in a way that doesn't add narrowing around >> fontification-functions, because as we can see it's not necessary in >> examples like this. > > If that is possible, sure. No one said that from now on every problem > in Emacs that causes slow responses will be handled by narrowing. But > if, for example, it turns out that the slow responses is due to time > it takes some code to traverse a long stretch of fontified buffer, > what other solution would you suggest except making the portion to be > traversed shorter? For all I know, the most optimal fix might still be implemented through narrowing, but it would be temporarily widened while fontificiation-functions are run. >>> If you dislike mis-fontification, turn font-lock mode off.  It's as easy >>> as that.  Mis-fontification is expected in such cases.  The docstring of >>> syntax-wholeline-max also mentions that "misfontification may then >>> occur". Why did you not protest at that time? >> >> I think we could have both speed and correctness, at least for files of >> this size. > > That is not a given, and the experience till now suggests otherwise. I have commented out the code which applies the narrowing in 'handle_fontified_prop' and recompiled. The result: - My 88K file is fontified correctly now. The redisplay and scrolling performance seem unaffected (meaning still fast). - dictionary.json (18M) seems to be fontified correctly as well now (it's a mess by default on master), its scrolling performance is unaffected too. The difference: I have to wait ~2 seconds the first time I press 'M->'. BTW, 'M-> M-<' triggers some puzzling long wait (~3 seconds) both on master and with my change, every time I issue this sequence of commands. diff --git a/src/xdisp.c b/src/xdisp.c index 099efed2db..02d7f6c562 100644 --- a/src/xdisp.c +++ b/src/xdisp.c @@ -4391,19 +4391,19 @@ handle_fontified_prop (struct it *it) eassert (it->end_charpos == ZV); - if (current_buffer->long_line_optimizations_p) - { - ptrdiff_t begv = it->narrowed_begv; - ptrdiff_t zv = it->narrowed_zv; - ptrdiff_t charpos = IT_CHARPOS (*it); - if (charpos < begv || charpos > zv) - { - begv = get_narrowed_begv (it->w, charpos); - zv = get_narrowed_zv (it->w, charpos); - } - narrow_to_region_internal (make_fixnum (begv), make_fixnum (zv), true); - specbind (Qrestrictions_locked, Qt); - } + /* if (current_buffer->long_line_optimizations_p) */ + /* { */ + /* ptrdiff_t begv = it->narrowed_begv; */ + /* ptrdiff_t zv = it->narrowed_zv; */ + /* ptrdiff_t charpos = IT_CHARPOS (*it); */ + /* if (charpos < begv || charpos > zv) */ + /* { */ + /* begv = get_narrowed_begv (it->w, charpos); */ + /* zv = get_narrowed_zv (it->w, charpos); */ + /* } */ + /* narrow_to_region_internal (make_fixnum (begv), make_fixnum (zv), true); */ + /* specbind (Qrestrictions_locked, Qt); */ + /* } */ /* Don't allow Lisp that runs from 'fontification-functions' clear our face and image caches behind our back. */