From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: JD Smith Newsgroups: gmane.emacs.bugs Subject: bug#71345: Feature: unleash font-lock's secret weapon; handle Qfontified = non-nil Date: Tue, 4 Jun 2024 08:08:40 -0400 Message-ID: References: <8A929E16-AF10-4D2B-AD71-AEAD4435F016@gmail.com> <1F2B8726-7594-494F-AB9D-08C48B7BCC43@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Content-Type: multipart/alternative; boundary="Apple-Mail=_31C5F096-BF60-490E-9693-F2184FE27400" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32665"; mail-complaints-to="usenet@ciao.gmane.io" Cc: dmitry@gutov.dev, 71345@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Jun 04 14:10:02 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sESzJ-0008KA-52 for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 04 Jun 2024 14:10:01 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sESz8-0002I5-TN; Tue, 04 Jun 2024 08:09:50 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sESz7-0002Hv-Az for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2024 08:09:49 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sESz7-0005by-3B for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2024 08:09:49 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sESzK-0006lX-3D for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2024 08:10:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: JD Smith Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 04 Jun 2024 12:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 71345 X-GNU-PR-Package: emacs X-Debbugs-Original-Cc: Dmitry Gutov , bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.171750295225851 (code B ref -1); Tue, 04 Jun 2024 12:10:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 4 Jun 2024 12:09:12 +0000 Original-Received: from localhost ([127.0.0.1]:44015 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sESyV-0006ir-ND for submit@debbugs.gnu.org; Tue, 04 Jun 2024 08:09:12 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:40068) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sESyT-0006ii-P4 for submit@debbugs.gnu.org; Tue, 04 Jun 2024 08:09:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sESyG-00029f-2M for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2024 08:08:56 -0400 Original-Received: from mail-il1-x133.google.com ([2607:f8b0:4864:20::133]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sESyE-0005Wm-1q for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2024 08:08:55 -0400 Original-Received: by mail-il1-x133.google.com with SMTP id e9e14a558f8ab-37491216a2bso14633675ab.1 for ; Tue, 04 Jun 2024 05:08:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717502933; x=1718107733; darn=gnu.org; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=arw7RADAmRYuMEsif4//EgrSbIwd5Z5A2kz4Mk28+so=; b=ntoEByyvemFllVGfDYRVHANlSMhO8mFs/gLo1WHNYLqAaf88GfI/G3DDsFJwysR2J6 rlEpCJ4OYAxuYzc/kf9iZHv1hpXNEGKOq2U6sDVhS/eWrYWIWeMK1qkch2w6HmxAZuxz ddLVYrkq+bWz/dY0nhVeYv855lRkAaCreuEKu6i7CrjuNB68sIyyQALkF70verZbZhXA FcQu5rJqNOBcuF42bEoQHSSr2TTS4kpXvDfeSgJ+DvRGxzeY7zx17080Zaly+pk9WlLq WzgRvWlP2YaZYX7foMsNQE8LLlwOWP0+SXGEcx/d2rVxNT/cQgtf1dEV3qbIMN3IQxdG ZwMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717502933; x=1718107733; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=arw7RADAmRYuMEsif4//EgrSbIwd5Z5A2kz4Mk28+so=; b=o+Dj5JhVSMd1N2aEut+QnE81TUIU5jcAlXiHjMZe3rIG6JRApzAXwk7rDIISSGUwSS i7RuCT7j2DdmP98ZPpcW3jIQB0gTthzyoRjlMm3D8e7rkIphFlWv2siNLiOhX4S/Axm/ PMt3cOzwKezGMa+NbknCk3TciFFVuzcs9/43rj8BNVEGZ+t5xXH53KmYlFJehSDXvIQL nDUVrtedoTkHqgt6NfzlyyIauvgoKQnaZqKwEpnQ8Q5mJUdPJyJkbvgugFoVZntAOp0M b5yN0bgI73kPIkPmt/36ZI6z8FNu6gp4rO+rSzzfpyjWFyGVayPIGT4yMvAnTAl9+Vrw B3Kw== X-Gm-Message-State: AOJu0YzXXWoUCgFdefCHiOVV0pKCbh7IjB0v5th9JAThzVD6KpIgOL4A VtfkcOdi0Vj/xN9ISosUaNdEV5y8XX+j8f+//Ph26p6hFrgkuM3f0rPppQ== X-Google-Smtp-Source: AGHT+IGn1c2ndvEhHvg1YOpoNPB5tOYcMt/U9bLHUd+P9GOZPUMraN2Ss2akUbQDWQAj4uOv999pOQ== X-Received: by 2002:a05:6e02:2185:b0:374:70ed:d741 with SMTP id e9e14a558f8ab-3748b96e54amr125917185ab.1.1717502932358; Tue, 04 Jun 2024 05:08:52 -0700 (PDT) Original-Received: from smtpclient.apple (cm-24-53-187-34.buckeyecom.net. [24.53.187.34]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3748a2052aesm22025825ab.40.2024.06.04.05.08.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Jun 2024 05:08:51 -0700 (PDT) In-Reply-To: X-Mailer: Apple Mail (2.3774.500.171.1.1) Received-SPF: pass client-ip=2607:f8b0:4864:20::133; envelope-from=jdtsmith@gmail.com; helo=mail-il1-x133.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:286534 Archived-At: --Apple-Mail=_31C5F096-BF60-490E-9693-F2184FE27400 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii >=20 >> That starts to sound like a lot of property slinging, which might = even >> dominate the work done. >=20 > Indeed, this amount of work could become significant. It's my main > worry, but I don't have a clear feel for how serious it would be > in practice. In my situation, the most likely scenario is that fontified=3Dnil is = noticed during redisplay when there is a fairly large stretch of = already-fontified property having the same value. So = jit-lock-fontify-now will quickly find a nice large chunk to call my = FONTIFICATION-FUNCTION=3DF-F with. Since jit-lock-after-change will likely clear away already-fontified and = set fontified=3Dnil, a single additional F-F on top of jit-lock-function = will probably be very well handled. A good question is how it would = scale with more functions all operating in the same region. One idea is = to rig up a test file, do some fake jit-lock-flushing on it, and check = performance of just subtracting/searching/dividing the already-fontified = property as you add more (fake) F-F's. For me, jit-lock-fontify-now of = a 2500 char chunk in a heavy treesitter buffer is in the 2-5ms range. = Individual F-F's could be much lighter weight. > We could try and unify `fontified` and `jit-lock-already-fontified` by > having a `fontified-done-value` variable and making the redisplay call > jit-lock whenever `fontified` has a value that's not-eq from > `fontified-done-value`. >=20 > So jit-lock would set `fontified-done-value` to the list of backends. Nice idea. Since redisplay just directs jit-lock to a "starting = position", it would be free to update however much buffer text it wants. = To overcome the issue of "many small domains", jit-lock could cheat, = for example checking if ANY chars in the next block need a particular = F-F, and running it on the full block if so. That's already implicitly = how jit-lock works now if I understand correctly. But things like = `text-property-any' will be quickly defeated by the combinatorics of a = large F-F set. Also, an advantage of keeping the fontified=3Dnil = semantics is that changes to jit-lock could be back-ported to earlier = Emacs versions. So here's an idea. You could invert the logic, and have a set of = `fontified-pending' properties which jit-lock-flush adds to as it sets = fontified=3Dnil, maintaining one property symbol for each F-F (e.g. = fontified-pending-N). Then jit-lock-flush's only job is to select and = apply one such property over the region, and fontify-now can use a = simple and very fast `text-property-any' as it loops through the list of = F-F's, and a final `remove-list-of-text-properties' to strip them all = away. For the convenience of jit-lock-after-change, setting a "single = property to rule them all" (fontified-pending=3Dt) directs jit-lock to = run all the F-F's, there. fontify-now could check for that first, then = only bother to look for the individual (-N) properties if it is not set. = Is there code out there that sets fontified=3Dnil on its own or is that = an internal detail of jit-lock? >> I imagine that the functions may also need a way to opt-out of = "deferred >> contextual refontification", for example if they add some other >> properties/overlays orthogonal to face. >=20 > `jit-lock.el` already has that info (it's the second arg to > `jit-lock-register`), but it currently doesn't keep track of it > individually for each backend. Great, so this information could be put into action using one of these = approaches. --Apple-Mail=_31C5F096-BF60-490E-9693-F2184FE27400 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii

That starts to = sound like a lot of property slinging, which might even
dominate the = work done.

Indeed, this amount of work could become = significant.  It's my main
worry, but I don't have a clear feel = for how serious it would be
in = practice.

In my = situation, the most likely scenario is that fontified=3Dnil is noticed = during redisplay when there is a fairly large stretch of = already-fontified property having the same value.  So jit-lock-fontify-now will quickly find a nice large chunk to call = my FONTIFICATION-FUNCTION=3DF-F = with.

Since jit-lock-after-change will = likely clear away already-fontified and set fontified=3Dnil, a = single additional F-F on top of jit-lock-function will probably be very = well handled.  A good question is how it would scale with more = functions all operating in the same region.  One idea is to rig up = a test file, do some fake jit-lock-flushing on it, and check performance = of just subtracting/searching/dividing the already-fontified property as = you add more (fake) F-F's.   For me, jit-lock-fontify-now of a 2500 = char chunk in a heavy treesitter buffer is in the 2-5ms range. =  Individual F-F's could be much lighter = weight.

We could try and = unify `fontified` and `jit-lock-already-fontified` by
having a = `fontified-done-value` variable and making the redisplay = call
jit-lock whenever `fontified` has a value that's not-eq = from
`fontified-done-value`.

So jit-lock would set = `fontified-done-value` to the list of = backends.

Nice idea. =  Since redisplay just directs jit-lock to a "starting position", it = would be free to update however much buffer text it wants.  To = overcome the issue of "many small domains", jit-lock could cheat, for = example checking if ANY chars in the next block need a particular F-F, = and running it on the full block if so.  That's already implicitly how = jit-lock works now if I understand correctly.  But things = like `text-property-any' will be quickly defeated by the combinatorics = of a large F-F set.  Also, an advantage of keeping the = fontified=3Dnil semantics is that changes to jit-lock could be = back-ported to earlier Emacs versions.

So = here's an idea.  You could invert the logic, and have a set of = `fontified-pending' properties which jit-lock-flush adds to as it sets = fontified=3Dnil, maintaining one property symbol for each F-F (e.g. = fontified-pending-N).  Then jit-lock-flush's only job is to select = and apply one such property over the region, and fontify-now can use a = simple and very fast `text-property-any' as it loops through the list of = F-F's, and a final `remove-list-of-text-properties' to strip them all away. =  For the convenience of jit-lock-after-change, setting a = "single property to rule them all" (fontified-pending=3Dt) directs = jit-lock to run all the F-F's, there. =  fontify-now could check for that first, then only bother to look = for the individual (-N) properties if it is not set.  Is there code = out there that sets fontified=3Dnil on its own or is that an internal = detail of jit-lock?

I imagine that the = functions may also need a way to opt-out of "deferred
contextual = refontification", for example if they add some = other
properties/overlays orthogonal to = face.

`jit-lock.el` already has that info (it's the = second arg to
`jit-lock-register`), but it currently doesn't keep = track of it
individually for each = backend.

Great, so this = information could be put into action using one of these = approaches.

= --Apple-Mail=_31C5F096-BF60-490E-9693-F2184FE27400--