Re: How to add pseudo vector types

[Stephen Leake <stephen_leake@stephe-leake.org>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stephen Leake <stephen_leake@stephe-leake.org>
>> Cc: Yuan Fu <casouri@gmail.com>,  monnier@iro.umontreal.ca,
>>   emacs-devel@gnu.org
>> Date: Tue, 20 Jul 2021 09:25:11 -0700
>> 
>> > In addition, Emacs records (for redisplay purposes) two places in each
>> > buffer related to changes: the minimum buffer position before which no
>> > changes were done since last redisplay, and the maximum buffer
>> > position beyond which there were no changes.  This can also be used to
>> > pass only a small part of the buffer to the parser, because the rest
>> > didn't change.
>> 
>> Again, the input to tree-sitter is a list of changes, not a block of
>> text containing changes.
>
> I fail to see the significance of the difference.  Surely, you could
> hand it a block of text with changes to mean that this block replaces
> the previous version of that block.  It might take the parser more
> work to update the parse tree in this case, but if it's fast enough,
> that won't be the problem.  Right?

tree-sitter doesn't store the previous text, so there's nothing to
compare it to. Alternately, this would require the parser to store the
previous text so it can compute the diff; that could be added in a
wrapper around tree-sitter.

wisi does store the previous text, so it could compute the diff. But
because of memory pressure, we want a design that does not require a
copy of the buffer text; when wisi is turned into an Emacs module, it
will not store a copy of the text.

>> If the parser is in an Emacs module, so it has direct access to the
>> buffer, then the hooks only need to record the buffer positions of the
>> insertions and deletions, not the new text. That should be very fast.
>
> (You are talking about the undo-list.)

Almost; the undo-list can get reset before the parser needs it. And
sometimes it is disabled. But it might make sense to try to use that
instead of maintaining a separate list of changes.

It might make sense to delete the matching change from the parser change
list when undo is invoked, rather than adding another change.

> But even this is wasteful: it is quite customary to delete, then
> re-insert, then re-delete again, etc. several times.  So collecting
> these operations will produce much more "changes" than strictly
> needed.  

Yes. The wisi parser Ada code includes a step that combines all the
changes (in arbitrary buffer-pos order) into a minimal list of changes
in buffer-pos order; that simplifies applying multiple changes to the
parse tree. We could move that to elisp, if that would help (it's in Ada
because I much prefer debugging Ada to debugging elisp). That could be
done in the buffer-change hook; if the current change can be combined
with the previous one, do that instead of adding a new one.

> That's why I'm trying to find a simpler, less wasteful strategies.
> Since TS is very fast, we can trade some of the speed for simpler,
> more scalable design of tracking changes.

I don't see how optimizing the change list makes it more "scalable"; the
worst case is that the optimal list is the complete list of actions the
user takes, and that will happen often enough to be an important case.

In practice font-lock is triggered on every character typed by the user
(Emacs is faster than people can type), so there will typically be only
one change; nothing to optimize.

In the case where some elisp is changing the buffer in several places
(ie indent-region, or some other re-format), optimizing the change list
might make sense, if the elisp code is not already optimized for that.

-- 
-- Stephe