From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Itai Berli Newsgroups: gmane.emacs.bugs Subject: bug#27525: 25.1; Line wrapping of bidi paragraphs Date: Thu, 20 Jul 2017 00:40:15 +0300 Message-ID: References: <8337abobuz.fsf@gnu.org> <87eftpa30a.fsf@blei.turtle-trading.net> <83a84djweb.fsf@gnu.org> <83shhsbakk.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="001a11443512b0024f0554b27c5d" X-Trace: blaine.gmane.org 1500500544 11593 195.159.176.226 (19 Jul 2017 21:42:24 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 19 Jul 2017 21:42:24 +0000 (UTC) To: 27525@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Jul 19 23:42:19 2017 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dXwjY-0002WY-1i for geb-bug-gnu-emacs@m.gmane.org; Wed, 19 Jul 2017 23:42:16 +0200 Original-Received: from localhost ([::1]:35113 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dXwja-00086e-E8 for geb-bug-gnu-emacs@m.gmane.org; Wed, 19 Jul 2017 17:42:18 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33146) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dXwjQ-00084l-2v for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2017 17:42:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dXwjL-0000SG-8G for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2017 17:42:08 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:45523) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dXwjL-0000S5-1n for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2017 17:42:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1dXwjK-0005oR-7L for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2017 17:42:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Itai Berli Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 19 Jul 2017 21:42:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 27525 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 27525-submit@debbugs.gnu.org id=B27525.150050046522274 (code B ref 27525); Wed, 19 Jul 2017 21:42:02 +0000 Original-Received: (at 27525) by debbugs.gnu.org; 19 Jul 2017 21:41:05 +0000 Original-Received: from localhost ([127.0.0.1]:48200 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dXwiO-0005nB-Oz for submit@debbugs.gnu.org; Wed, 19 Jul 2017 17:41:05 -0400 Original-Received: from mail-wm0-f50.google.com ([74.125.82.50]:38449) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dXwiM-0005ma-KP for 27525@debbugs.gnu.org; Wed, 19 Jul 2017 17:41:03 -0400 Original-Received: by mail-wm0-f50.google.com with SMTP id w191so10520647wmw.1 for <27525@debbugs.gnu.org>; Wed, 19 Jul 2017 14:41:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=ATCZ6vINtdXcrHpzxrrrMb2Z5N9EAmm3G+P6GyOK9QU=; b=DEC0AtzWm6dsjVHU/6Jo//DGHB8DVfAl+X0Bk335RbbnObQ+f84ZMb8UBK7EnxCLFn MdhEyschGT+l30oVCPlsCp4wG2OnTr+QiSV1HN9QnGjKdvYvopMkNjlXUljP7TVfgyyy +zboMF2NUU7+EJEDeDSLtV1pvm40cbKR5DISJA+9cq4d3fjVkH6k4vy9Jskd1Rzd6ldU BkCQxW4nXzNOh8QThfXUcPk7dD3hRP8MQ6CSyY45V5d9TdrxJZbFxzxZmVFJZoicIjI8 fNiPvYMzRO3g716QI0f7Gvw5NOoSBGZjH/cuquSHkRF7+s2FrfE8f3GSIi3R/tOf9oxp WI2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=ATCZ6vINtdXcrHpzxrrrMb2Z5N9EAmm3G+P6GyOK9QU=; b=Z23bEz5KL7eXPeP1JZk5zgz7EGrGL0qNscioGEwNgFV2I3/yhh3qX2CQWpxuNRjFSj 9xLkpdL7CsWKsR1nX4vqhNA6SVJh7eFZd0dXH/fJN66law2cxpUl8xGG3z9kHy2q5k15 uXb7F35Vim5wl4o/ZMLUFikWDtPwNEU5X9yRGf8mdez5obIV5BrgINL+zNz0GB+YJ8XX Aff91wMGH/JJAoJkwWiacevEM3hQrrOqZ42tXZM5viyUEZbwFTJFqUZdRPOtWIwi4QRK NPVdN7yVJQ7n8JL5oo1YJ6p8MhFDslTvIujD2meogwdxnInQO3myqI9rEGcDzc6DwKCO /UEQ== X-Gm-Message-State: AIVw1105OMek+Fxutz00mul/HkcpnrejXOgY49xeCAhsmyN4td7MN5TK WRwS9zp0iu4YvGUc9RuvbCuja6jaZJng X-Received: by 10.28.7.211 with SMTP id 202mr881730wmh.113.1500500456372; Wed, 19 Jul 2017 14:40:56 -0700 (PDT) Original-Received: by 10.28.197.9 with HTTP; Wed, 19 Jul 2017 14:40:15 -0700 (PDT) In-Reply-To: <83shhsbakk.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:134767 Archived-At: --001a11443512b0024f0554b27c5d Content-Type: text/plain; charset="UTF-8" I appreciate your hard work. I can imagine that it took you a lot of time, effort and agony to pull off this feature, and your efforts were worthwhile: you created a good and helpful feature. For some purposes the feature is perfect as it is, especially now that you've allowed customization of the paragraph separator. I believe Emacs can be the #1 plain-text bidi editor out there, but this hinges on fixing this bug. > I maintain that Emacs deviates from the UBA in a relatively minor way, in an aspect that is only tangentially related to reordering bidirectional text for display, and that raises its head in situations that are relatively rare in practice, and in many of those rare cases can be easily worked around by breaking long lines. One of the valuable aspects of an ISO standard is that it is not left to the personal judgment of a programmer to decide what is worth implementing, and how to do so. It is not for you to decide what is a minor detail and what is a major one, what is tangential and what is core. You need to implement it to the letter, or else you can't claim conformance, no matter how slight you imagine your deviation to be. On what do you base your claim that this problem occurs relatively rarely in practice? This is the kind of statement that only a specialist linguist/statistician can make. And have you taken into account the type of demographics who use Emacs' bidi feature and the kinds of texts they're likely to type? Contrary to what you said, my personal experience show that this is a major inconvenience, and that it is a situation that occurs very often, almost every paragraph, in fact, since I write primarily LaTeX documents where English markup is intermixed with predominantly Hebrew text containing frequent quotes from English textbooks and articles. Yes, breaking lines is a possible workaround for LaTex, but it makes for ugly and erratic looking paragraphs that are difficult to read and edit. And what about documents that are not LaTeX? What workaround do they have? You mention breaking "long lines", but this is not just a problem of long lines. It takes just two English words inside a Hebrew paragraph that happen to fall on a line break, to manifest this bug. > even today, 10 years later, still shines among all the bidi-aware editors out there, certainly among those of the Free Software variety. Yes, Emacs shines as one of the very rare bidi-aware text editors that enable entering explicit directional formatting characters. This is indeed to Emacs' credit and is a very helpful feature. However, Emacs also shines as possibly the only bidi-aware text editor that botches the line wrapping of bidi paragraphs. Every single editor that I've checked gets it right: from Word to Kate to GEdit to Google Docs to BlueFish to TextEdit. > And I don't really understand what is the purpose of your insistence on the formal definition of this deviation. It certainly won't help fixing this issue any time soon, not unless someone steps forward to do the job, which IMO is quite large. I don't know what you mean by 'the formal definition of this deviation'. I think that Emacs should not mislead the users and potential users. That it should not claim to conform to a standard when it does not. I think that when prospective users google "Emacs bidi" or "Emacs unicode" they will be able to easily see that there's a problem with bidi line wrapping and that if they require a text editor that is Unicode compliant they should look elsewhere. The keywords are: transparency, truth in advertising, user-friendly, and standards-oriented. > The Emacs manual already describes this deviation. In the online manual sections 22.19 (Bidirectional Editing) and 37.26 (Bidirectional Display) claim that Emacs implements the Unicode Bidirectional Algorithm. On Wed, Jul 19, 2017 at 8:28 PM, Eli Zaretskii wrote: > > From: Itai Berli > > Date: Wed, 19 Jul 2017 15:59:14 +0300 > > > > The Emacs manual and all official Emacs publications should make it > clear that Emacs does not conform to > > the Unicode Standard. Anything else is simply not true, and is a > deliberate misleading. > > The Emacs manual already describes this deviation. > --001a11443512b0024f0554b27c5d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I appreciate your ha= rd work. I can imagine that it took you a lot of time, effort and agony to = pull off this feature, and your efforts were worthwhile: you created a good= and helpful feature. For some purposes=C2=A0the feature is perfect as it is, especially now that you've a= llowed customization of the paragraph separator. I believe Emacs can be the= #1 plain-text bidi editor out there, but this hinges on fixing this bug.

> I main= tain that Emacs deviates from the UBA in a relatively minor way,
=
in an aspect that is only tangential= ly related to reordering
bidirectional text for display, and that raises its h= ead in situations
that are relatively rare in practice, and in many of those rar= e cases
can be easily worked around by breaking long lines.

One of the valuable aspects of an ISO standard is that it is not left t= o the personal judgment of a programmer to decide what is worth implementin= g, and how to do so. It is not for you to decide what is a minor detail and= what is a major one, what is tangential and what is core. You need to impl= ement it to the letter, or else you can't claim conformance, no matter = how slight you imagine your deviation to be.

On what do you base your claim that this= problem occurs relatively rarely in practice? This is the kind of statemen= t that only a specialist linguist/statistician can make. And have you taken= into account the type of demographics who use Emacs' bidi feature and = the kinds of texts they're likely to type?

Contrary to what you said, my personal experience show that this is a maj= or inconvenience, and that it is a situation that occurs very often, almost= every paragraph, in fact, since I write primarily LaTeX documents where En= glish markup is intermixed with predominantly Hebrew text containing freque= nt quotes from English textbooks and articles.

Yes, breaking lines is a possib= le workaround for LaTex, but it makes for ugly and erratic looking paragrap= hs that are difficult to read and edit. And what about documents that are n= ot LaTeX? What workaround do they have?

You mentio= n breaking "long lines", but this is not just a problem of long l= ines. It takes just two English words inside a Hebrew paragraph that happen= to fall on a line break, to manifest this bug.

&g= t;=C2=A0=C2=A0even today, 10 years later, still shines among all thebidi-aware editors out there, certainl= y among those of the Free
Software variety.

Yes, Emacs = shines as one of the very rare bidi-aware text editors that enable entering= explicit directional formatting characters. This is indeed to Emacs' c= redit and is a very helpful feature.

However,= Emacs also shines as possibly the only bidi-aware text editor that botches= the line wrapping of bidi paragraphs. Every single editor that I've ch= ecked gets it right: from Word to Kate to GEdit to Google Docs to BlueFish = to TextEdit.

<= /div>
>=C2=A0And I don't really understand what is the purpose of = your insistence
on the formal = definition of this deviation.=C2=A0 It certainly won't help
fixing this issu= e any time soon, not unless someone steps forward to
do the job, which IMO is qu= ite large.=C2=A0

I don't know what you mean by = 'the formal definition of this deviation'. I think that Emacs shoul= d not mislead the users and potential users. That it should not claim to co= nform to a standard when it does not. I think that when prospective users g= oogle "Emacs bidi" or "Emacs unicode" they will be able= to easily see that there's a problem with bidi line wrapping and that = if they require a text editor that is Unicode compliant they should look el= sewhere. The keywords are: transparency, truth in advertising, user-friendl= y, and standards-oriented.

>= =C2=A0The Emacs manual already describes t= his deviation.

In the online manual sections= =C2=A022.19 (Bidirectional Editing) and=C2=A037.26 (Bidirectional Display) = claim that Emacs implements the=C2=A0Unicode Bidirectional Algorithm= .


On Wed, Jul 19, 201= 7 at 8:28 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> From: Itai Berli <itai.berli@gmail.com>=
> Date: Wed, 19 Jul 2017 15:59:14 +0300
>
> The Emacs manual and all official Emacs publications should make it cl= ear that Emacs does not conform to
> the Unicode Standard. Anything else is simply not true, and is a delib= erate misleading.

The Emacs manual already describes this deviation.

--001a11443512b0024f0554b27c5d--