bug#27525: 25.1; Line wrapping of bidi paragraphs

[Itai Berli <itai.berli@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4452 bytes --]

And in case you counter that Emacs takes advantage of the "higher-level
protocol" clause, this clause doesn't apply to paragraph 3.4, which Emacs
violates.

But don't take my word for it. I contacted Mr. Aharon Lanin in the matter.
Mr. Lanin is a senior software engineer at Google Tel Aviv as well as one
of the three editors of the Unicode Bidirectional Algorithm from version
6.3.0 till the latest one, v. 10.0.0. I presented him with the following
screenshot of a bidi paragraph in Emacs (it's the same screenshot as I
posted originally in the present ticket): http://imgur.com/Bckn7zP

I list below an excerp from our conversation, which Mr. Lanin has given me
permission to quote.

--- EXCERPT BEGIN ---

***Me***: Just to be clear, are the following statements correct for the
Unicode Standard v. 8.0.0 and above? (I'm mentioning v. 8.0.0 because this
is the version that Emacs claims conformance to.)

1. The way Emacs handles line wrapping of bidi paragraphs does not satisfy
section 3.4 of the Unicode Bidirectional Algorithm. There are no provisions
for higher-level protocol interpretation of this section.

2. If a candidate implementation of the Unicode Bidirectional Algorithm
doesn't satisfy section 3.4, it does not conform to the Unicode
Bidirectional Algorithm.

3. If a candidate implementation of the Unicode Standard does not conform
to the Unicode Bidirectional Algorithm (of the same version), it does not
conform to the Unicode Standard.

***Lanin*** I think so, but I am a programmer, not a lawyer :-)

--- EXCERPT END ---

The Emacs manual and all official Emacs publications should make it clear
that Emacs does not conform to the Unicode Standard. Anything else is
simply not true, and is a deliberate misleading.

On Wed, Jul 19, 2017 at 11:50 AM, Itai Berli <itai.berli@gmail.com> wrote:

> Eli, in different bug report, namely 27526, I recently wrote the following
> remark:
>
> > the line-wrapping bug is still a major annoyance, at best, and until it
> is fixed, Emacs cannot claim to be Unicode compliant.
>
> to which you replied:
>
> > I disagree, as I already said many times.
>
> You do agree, though, that Emacs does not conform to the Unicode
> Bidirectional Algorithm as specified in the Unicode Standard Annex #9.
> After all, the following paragraph appears in the bidi code itself (
> http://git.savannah.gnu.org/cgit/emacs.git/tree/src/bidi.c):
>
>    Note that, because reordering is implemented below the level in
>    xdisp.c that breaks glyphs into screen lines, we are violating
>    paragraph 3.4 of UAX#9. which mandates that line breaking shall be
>    done before reordering each screen line separately.
>
> So the only thing you disagree with me is that non-conformance to the
> Unicode Bidirectional Algorithm is tantamount to non-conformance to the
> Unicode Standard. Well, this disagreement is easily settled by reading
> article C12 'Bidirectional Text' of section 3.2 'Conformance Requirements'
> of the Unicode Standard:
>
> A process that displays text containing supported right-to-left characters
> or embedding codes shall display all visible representations of characters
> (excluding format characters) in the same order as if the Bidirectional
> Algorithm had been applied to the text, unless tailored by a higher-level
> protocol as permitted by the specification.
>
> * The Bidirectional Algorithm is specified in Unicode Standard Annex #9,
> “Uni- code Bidirectional Algorithm.”
>
>
> On Sun, Jul 9, 2017 at 9:30 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Benjamin Riefenstahl <b.riefenstahl@turtle-trading.net>
>> > Cc: 27525@debbugs.gnu.org,  itai.berli@gmail.com
>> > Date: Sun, 09 Jul 2017 20:17:41 +0200
>> >
>> > >> From: Richard Stallman <rms@gnu.org>
>> > >> It might be good if Emacs could refill lines automatically the way
>> > >> some other ediors do.
>> >
>> > Eli Zaretskii writes:
>> > > We already have that: "M-x visual-line-mode RET".
>> >
>> > JFTR, even that does not help in this case.  With visual-line-mode the
>> > order of the lines is still wrong with the text that the OP gave.
>>
>> Of course.  It isn't supposed to help.  From the POV of the display
>> engine, visual-line-mode is just a fancy kind of producing
>> continuation lines, so all the problems you see with continued lines
>> will still be there in visual-line-mode.
>>
>
>

[-- Attachment #2: Type: text/html, Size: 7719 bytes --]