Re: merge emacs-bidi into the main tree

["Eli Zaretskii" <eliz@elta.co.il>]

> Date: Tue, 12 Aug 2003 10:49:26 +0900 (JST)
> From: Kenichi Handa <handa@m17n.org>
> 
> In brief, what I did in emacs-bidi are:
> 
> (1) change xdisp.c to call get_next_display_element_visually
>   and set_iterator_to_next_visually instead of
>   get_next_display_element and set_iterator_to_next.
> 
> (2) make a new file bidi.c that implements
>   get_next_display_element_visually and
>   set_iterator_to_next_visually.
> 
> (3) make a new file bidi.el that implements simple
>   bidi-reordering function that is called from
>   get_next_display_element_visually to create a cache in
>   (struct it).
> 
> (4) change xterm.c to display glyphs flushing to right when
>   orientation-reversed is non-nil.
> 
> My current plan is to replace (2) and (3) with Eli's code.

It's not gonna be that simple ;-)

The code I wrote is designed to be plugged into the part of the
display engine that walks the buffer and, for each character it
encounters, generates a `struct glyph' element of the glyph matrix.
Currently, ignoring overlays, text properties, etc., this buffer scan
is linear, from left to right, and going to the next buffer position
is simple:

	IT_BYTEPOS (*it) += it->len;
	IT_CHARPOS (*it) += 1;

(this is from xdisp.c:set_iterator_to_next).  That is, we simply
increment the buffer's character position by one.  (If the next
display element comes from something other than the buffer, the code
in set_iterator_to_next does similar things with the object that is
the source of characters.)

The code I wrote replaces the buffer position increment with a call to
a function.  Thus, the above two lines should roughly be replaced with
this:

	bidi_get_next_char_visually (it->bidi_it);

[IT_BYTEPOS (*it) and IT_CHARPOS (*it) get set to the right values
inside that call.]

What bidi_get_next_char_visually does is to find the next character in
the buffer in the visual order; that could involve quickly scanning
across a run of characters until we find the one we want to display.
For example, i fthe buffer's contents are

	abcdABCDefg

then after processing `d', the code will scan until `D', then go back
to `C', `B', and `A', in that order, then jump to `e', and finally
proceed to `f' and `g'.

(To save future work, some crucial information about the characters
over which we scan is cached inside the bidi_it struct, to facilitate
processing later when we need to go back to those characters and
generate the glyph matrix elements from them.  So, in the above
example, going back from 'D' to 'A' boils down to delivering
information already cached during the forward scan.)

So a linear scan becomes non-linear, but the details of that are
completely concealed from set_iterator_to_next; as far as it is
concerned, it just magically jumped over a few characters and landed
on the next character to be displayed.  The glyph matrices are layed
in the visual order.

In other words, the structure and logic of the code in xdisp.c remains
largely unchanged, the only changes being where we find the position
of the next display element.  Or so I hope, anyway...

(There still needs to be more code in the terminal-specific parts of
Emacs, that displays the glyphs either starting at the left or the
right margin of the screen/window, depending on the current
paragraph's base direction; the latter gets decided by subroutines of
bidi_get_next_char_visually and is stored in the bidi_it structure.  I
believe this code, at least for the X terminals, already largely
exists in m17n.org's emacs-bidi.)