bug#23501: Non-regex-based syntax highlighting

[Stefan Kangas <stefan@marxist.se>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Nir Friedman <quicknir@gmail.com>
>> Date: Tue, 10 May 2016 16:16:03 -0400
>> Cc: 23501@debbugs.gnu.org
>>
>> My idea for a hook was basically to make it possible to provide a callback function to the Major mode. If this
>> callback function is provided, then when a new file is loaded or an existing one saved with modifications, the
>> callback function is called with the full path to the file.
>
> The syntax highlighting should change also when you modify the buffer,
> not only when you save it.  How will that work with your proposed hook?
>
>> The callback function must return something that
>> basically tells the major mode how to color everything. A simple way would just be to return a list of the colors
>> for every single non-whitespace character taken sequentially. A single very fast pass through this list would
>> then be able to color every character.
>
> The hook cannot return a color, because the colors are defined via
> faces.  It should return faces instead.
>
>> Is there a reason why that would not be workable?
>
> Maybe it is workable, but you are missing too many details of how
> syntax highlight works in Emacs.  As I wrote previously, I encourage
> you to study how that works, in order for the proposal to be workable
> and practical.
>
>> Also, can you point me to where exactly (e.g. via link to the
>> emacs github mirror) the major modes are stored?
>
> It's not the major mode that you need to look at, it's the font-lock
> machinery.  Major modes just use the font-lock features by setting the
> font-lock faces on portions of the buffer.  Then at display time, the
> visible portion of the buffer are displayed as specified by those
> faces.  You will see that each major mode simply sets the font-lock
> faces, and leaves the rest to the core features.
>
> See font-lock.el and font-core.el for the font-lock features, and
> jit-lock.el for the JIT coloring of the visible portions of the
> buffer.

It seems like there was a proposal here 4 years ago, that Eli said was
unworkable.  There were also some outstanding questions regarding said
proposal.  But there were no further updates after that.

I'm therefore closing this bug report now.  If anyone wants to continue
working on something along these lines, feel free to reopen this bug
report or file a new one.

Thanks.

Best regards,
Stefan Kangas