Re: Compositions and bidi display

[Eli Zaretskii <eliz@gnu.org>]

> From: Kenichi Handa <handa@m17n.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 03 May 2010 11:39:24 +0900
> 
> > This is only the simplest case, with just 2 embedding levels: the base
> > level of the paragraph, and the (higher) level of the embedded R2L
> > text.  The general case is much more complex: there could be up to 60
> > nested levels, and some of them could begin or end at the same buffer
> > position.  bidi.c handles all this complexity by means of a very
> > simple algorithm, but that algorithm needs to know a lot about the
> > characters traversed so far.  I don't think exposing all these
> > internals to xdisp.c is a good idea.
> 
> Just exposing (or creating) one function that tells where
> the current bidi-run ends is enough.  Is it that difficult?

Maybe not, but what will this solve?  The end of a level run can still
potentially be far away, much farther than we need to look to find
compositions.  I'm trying to find a way of searching smaller parts of
the buffer.

In addition, going back in the buffer is much less efficient than
going forward, so it's probably a good idea to avoid looking back by
decrementing buffer positions.

> > > Note that composition_compute_stop_pos just finds a stop
> > > position to check, and the actual checking and composing is
> > > done by composition_reseat_it which is called by
> > > CHAR_COMPOSED_P.
> 
> > Right, but the same is true for the bidi iteration: I need only to
> > know when to check for composition; the actual composing will be still
> > done by composition_reseat_it.  I just cannot assume that I always
> > move linearly forward in the buffer.  Therefore, it is not enough to
> > have only the next stop position recorded in the iterator.  I need
> > more information recorded.  What I'm trying to determine in this
> > thread is what needs to be recorded and how to compute what's needed.
> > Thanks for helping me.
> 
> I don't understand the logic of "Therefore" in the above
> paragraph.

When we traverse the buffer in a single direction, like with Emacs 23
redisplay, we only need to record the single next position to check
for compositions, which is always _after_ (at higher buffer position)
than where we are.  Until we get to that position, we _know_ there
will be no composition sequences in the buffer.

By contrast, when we traverse the buffer non-linearly, changing
direction and jumping back and forth, we can suddenly find ourselves
beyond this single next position, without actually passing it and
handling the composition at that position.  So we need to record more
information about possible places of compositions in the buffer, to
account for such non-linear movement.

> > > > Another idea would be to call composition_compute_stop_pos repeatedly,
> > > > starting from the last cmp_it->stop_pos, until we find the last
> > > > stop_pos before the current iterator position, then compute the
> > > > beginning and end of the composable sequence at that position, and
> > > > record it in the iterator.  Then we handle the composition when we
> > > > enter the sequence from either end.
> > > 
> > > To move from one composition position to the next, we must
> > > actually call autocmp_chars and find where the current
> > > composition ends, then start searching for the next
> > > composition.  As autocmp_chars calls Lisp and all functions
> > > to compose characters, it's so inefficient to call it
> > > repeatedly just to find the last one.
> 
> > If the buffer or string is full of composed characters, then yes, it
> > would be a slowdown.  Especially if the number of ``suspect'' stop
> > positions is much larger than the number of actual composition
> > sequences.  But what else can be done, given the design of the
> > compositions that doesn't let us know the sequence length without
> > actually composing the character?
> 
> Isn't it faster to call bidi_get_next_char_visually
> repeatedly.  At least it doesn't call Lisp.

I'm confused.  bidi_get_next_char_visually is what we use now to move
through the buffer, so using it gets me back at the problem I'm trying
to solve: how to know, at an arbitrary position returned by
bidi_get_next_char_visually, whether it is inside a composition
sequence.

What am I missing?