Re: bidi-display-reordering is now non-nil by default

[Eli Zaretskii <eliz@gnu.org>]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: larsi@gnus.org,
>     list-general@mohsen.1.banan.byname.net,
>     emacs-devel@gnu.org
> Date: Thu, 04 Aug 2011 19:04:22 +0900
> 
>  > Nor is it specific to text external to Emacs: after all, the
>  > internal storage of text in Emacs, as in many other applications,
>  > is just a linear byte stream.
> 
> That's just saying that you *can* use directional marks for this
> purpose.  It is not a reason you *should*.

Directional marks need to be supported anyway, if we want to be
compliant.  Using that for "fixing" specially-formatted text as well
gives us 2 features for the cost of one.

> I would not extrapolate from the MULE experience to designs that could
> be a lot more precise and explicit.

Precisely and explicitly, I think that using lossy conversions is not
a good idea, if there are practical alternatives.  Just visiting a
file, then saving it without changes should produce the a byte stream
that is identical to the original one.  I hope you agree with that.

>  >   . since each character can have only one value of the direction
>  >     property, you cannot do this in any simple way; you'd need to
>  >     split the original region in 3 parts, which is ugly
> 
> It's no uglier than inserting control sequences

I mean ugly on the implementation level.  This splitting (and the
corresponding joining) need to be implemented, or else each Lisp
program would need to do it by hand.  By contrast, inserting a
character does not need any additional features.  Text in mail summary
buffers is already formatted specially, by inserting separators
between fields; adding one more character to the field separator is
not fundamentally different.

> which means that text has yet another feature that prevents it from
> being an array of characters.

??? LRM is just another character, like SPC or `|'.  The text is still
an array of characters after inserting it.

> I'm still not convinced that the problems Lars worries about won't
> annoy the users.

I'm not convinced either.  But "the proof of the pudding is in
eating."  These questions are around since 12 years ago, and no one
ever had the definitive answers.  If I'd to wait for such answers,
Emacs wouldn't have bidi support it has now.  So I made some decisions
(and published them here almost 2 years ago, btw), and implemented
them.  Having used them for some time, I can say that they are quite
satisfactory, but that's me.

Let's get Emacs 24.1 out the door, using the existing functionality,
and let the users of R2L scripts at large judge.  This discussion,
however useful and enlightening, cannot be a substitute for user
experience, especially since most of the participants don't use R2L
scripts.

>  > If we drop the marks on yanking, text will look differently when
>  > yanked, sometimes completely differently, to the degree of being
>  > incomprehensible.
> 
> Obviously.  Nevertheless, I suspect there will be applications where
> that is desirable (especially for marks at the boundary of the yanked
> region).

I don't see why, but it isn't worth arguing, since removing them is
easy.