On Jun 4, 2024, at 5:52 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
Also, I think it's worth distinguishing the API from its implementation.
Fair point.
or on the contrary if it's careful to do something like
(jit-lock-flush (min my-pd-old-beg (my-pd-new-beg-estimate))
(max my-pd-old-end (my-pd-new-end-estimate))
#'my-pd-backend)
I don't need to estimate, I have treesitter to tell me precisely!
Agreed that could be an issue. In practice keyword-based fontification can
lead to these same sorts of conflicts for non trivial FACE forms too.
So backends would need to ensure the changes they are making in the buffer
are interoperable with the other likely backends (in particular font-lock).
Because a given buffer can have several (window-)points,
position-dependent highlighting will ideally want to be added via
(window-specific) overlays rather than text-properties.
In general, yes. In my case having the scope be per-buffer not per window makes the most sense in terms of the functionality.
This means other backends would still need to add to
font-lock-extra-managed-props any unusual properties they will apply
(or do the equivalent on their own during unfontify).
No: `font-lock-extra-managed-props` is a variable that belongs to
font-lock, not jit-lock. `font-lock` is *one* backend of jit-lock.
I tend to associate font-lock and after-change in my head, but that's backwards. Presumably after-change would clobber fontified and whatever additional properties jit-lock uses to track its backends. So font-lock has no special status. Perhaps I tend to think of it as special because it touches so many locations in the buffer.
Other backends will have to use something else (and should use other
text-properties or should use overlays (or make sure font-lock is not
used), otherwise you'll get into conflicts with font-lock).
Sometimes those are unavoidable, if you need to add some 'face on top of what font-lock did.
If we want to allow backends that are not independent (e.g. your PD
highlighting that has to run after font-lock), then it make the API more
complex, since `jit-lock-flush` on the font-lock backend would have to
know to also flush the PD backend
I think it will be limiting to insist that all backends must be fully orthogonal (i.e. can apply in any arbitrary order), so some sort of priority or ordering system seems important. That said, mine only has to run over font-lock because of multi-line strings/comments, and the inability to set an independent 'face property.
It's actually too bad font-lock doesn't itself use font-lock-face for much, despite what the docs say [1]. Since it sets 'face directly, it overrides any other property alias for 'face. If it instead used 'font-lock-face (or left that to modes, and used some higher-ranked 'font-lock-priority-face), other backends could use their own 'alternate-face, and the order in char-property-alias-alist would set the priority. Then it would be much easier to make face-using backends more orthogonal.
Regarding the priority of backends, while looking through jit-lock, I was reminded of the completion-at-point-function's API, and the ability to claim :exclusive access, halting calls to further functions on the list. That's a strong power, but if used wisely, could be effective.
I think the code would loop over chunks of text where the property is
`eq` (e.g. using `next-single-property-change`). Then within each such
chunk of text it would iterate over either all backends (and skip
those mentioned in the `already-fontified` text-property) or over the
`pending` property backends. That seems easy enough.
But those may be small regions which are inefficient to work on. This as you say is an implementation detail so probably not important yet.