Re: [SPAM UNSURE] Re: Reliable after-change-functions (via: Using incremental parsing in Emacs)

[Eli Zaretskii <eliz@gnu.org>]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Fri, 03 Apr 2020 10:11:05 -0800
> 
> > Emacs represents buffer text as a superset of UTF-8, with the
> > violations of strict UTF-8 being very rare in buffers that hold
> > program sources.  The function we can provide that lets tree-sitter
> > access buffer text can cope with those violations,
> 
> Ok. "cope with those violations" = "do some encoding".

If we use "encoding" terminology for this, it will be confusing and
will cause misunderstandings.  "Conversion" is better, IMO.  Some
sequences may need to be converted when feeding them to tree-sitter.

But I think tree-sitter should be able to cope with this itself.  It
is unreasonable to expect strict UTF-8 from all applications.  Maybe
I'm dreaming, but ISTR there is (or was) an issue on their issue
tracker about this.

> We cannot avoid encoding (either in the read function provided to
> tree-sitter, or in the tree-sitter lexer), but the encoding may be very
> simple and efficient.

Once again, please reserve "encoding" to the likes of
encode-coding-region or code_convert_string, to avoid confusion.