bug#27525: 25.1; Line wrapping of bidi paragraphs

[Eli Zaretskii <eliz@gnu.org>]

> From: Itai Berli <itai.berli@gmail.com>
> Date: Thu, 20 Jul 2017 00:40:15 +0300
> 
> I believe Emacs
> can be the #1 plain-text bidi editor out there, but this hinges on fixing this bug.

And I believe you exaggerate the importance of this issue, and how
much it diminishes the usefulness of the Emacs bidi support.  Can we
agree to disagree about that, now that we've reiterated that
disagreement many times, and all of that is forever recorded in the
bug tracker?

> 
> > I maintain that Emacs deviates from the UBA in a relatively minor way,
> in an aspect that is only tangentially related to reordering
> bidirectional text for display, and that raises its head in situations
> that are relatively rare in practice, and in many of those rare cases
> can be easily worked around by breaking long lines.
> 
> One of the valuable aspects of an ISO standard is that it is not left to the personal judgment of a programmer
> to decide what is worth implementing, and how to do so. It is not for you to decide what is a minor detail and
> what is a major one, what is tangential and what is core. You need to implement it to the letter, or else you
> can't claim conformance, no matter how slight you imagine your deviation to be.

Of course, it's for me to decide.  Emacs is not an implementation of
the Unicode Standard: Emacs _follows_ the Unicode recommendations
where we decide it to be useful/practical, and doesn't where we don't.

> On what do you base your claim that this problem occurs relatively rarely in practice? This is the kind of
> statement that only a specialist linguist/statistician can make. And have you taken into account the type of
> demographics who use Emacs' bidi feature and the kinds of texts they're likely to type?

It doesn't take a statistician/linguist to realize that

  . long lines that wrap on Emacs display are rare to begin with
  . lines with predominantly RTL text in LTR paragraphs are rare, and
    likewise lines with predominantly LTR text in RTL paragraphs
  . multiplying 2 rare cases makes the result very rare

> Contrary to what you said, my personal experience show that this is a major inconvenience, and that it is a
> situation that occurs very often, almost every paragraph, in fact, since I write primarily LaTeX documents
> where English markup is intermixed with predominantly Hebrew text containing frequent quotes from English
> textbooks and articles.

LaTeX documents can easily work around the problem by breaking long
lines into shorter ones.

> Yes, breaking lines is a possible workaround for LaTex, but it makes for ugly and erratic looking paragraphs
> that are difficult to read and edit.

I fail to see why it would be ugly or hard to read.  Especially since
you can now have a different paragraph direction after every newline.
Perhaps you need to break lines more judiciously, not at random
points.

> And what about documents that are not LaTeX? What workaround do they
> have?

Plain text documents, and documents that are "nearly plain text", like
TeX, Texinfo, and other similar systems, rarely if ever consider
newlines as significant.  So this workaround is available there as
well.  About the only exception I know of is poetry, where over-long
lines are even rarer.

Btw, on GUI terminals there's one other workaround: make your Emacs
window wider.  That works with any file/buffer, not just text-like
ones.

> You mention breaking "long lines", but this is not just a problem of long lines. It takes just two English words
> inside a Hebrew paragraph that happen to fall on a line break, to manifest this bug.

Yeah, and how frequently does that happen?

> However, Emacs also shines as possibly the only bidi-aware text editor that botches the line wrapping of bidi
> paragraphs. Every single editor that I've checked gets it right: from Word to Kate to GEdit to Google Docs to
> BlueFish to TextEdit.

You are free to use those other bidi-aware editors, if they suit your
needs better.  They don't have half the bidi features you get in
Emacs, but if line-wrapping is so much more important for you than all
the rest of the UBA, you don't have to use Emacs.

> > The Emacs manual already describes this deviation.
> 
> In the online manual sections 22.19 (Bidirectional Editing) and 37.26 (Bidirectional Display) claim that Emacs
> implements the Unicode Bidirectional Algorithm.

You have the latest sources in the Git repository you cloned, look
there for the latest text.

Once again: this is an annoyance, and I'd love to see it fixed.  But
it's a minor annoyance, which happens rarely, and on most cases there
are workarounds.  Fixing it is a large job, and will take a motivated
volunteer with a lot of talent or a lot of free time (or both).  Until
we are lucky to have that, we will have to live with this annoyance.

Cane we PLEASE finally agree to disagree about this?  I see no reason
for discussing this further, as we are just repeating the same
arguments again and again.