From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#27544: 25.1; Visualization of Unicode bidirectional marks Date: Sat, 01 Jul 2017 13:36:24 +0300 Message-ID: <83r2y0php3.fsf@gnu.org> References: Reply-To: Eli Zaretskii NNTP-Posting-Host: blaine.gmane.org X-Trace: blaine.gmane.org 1498905444 11394 195.159.176.226 (1 Jul 2017 10:37:24 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 1 Jul 2017 10:37:24 +0000 (UTC) Cc: 27544@debbugs.gnu.org To: Itai Berli Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Jul 01 12:37:18 2017 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dRFm6-0002cJ-R8 for geb-bug-gnu-emacs@m.gmane.org; Sat, 01 Jul 2017 12:37:15 +0200 Original-Received: from localhost ([::1]:53693 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dRFmC-0006RW-7a for geb-bug-gnu-emacs@m.gmane.org; Sat, 01 Jul 2017 06:37:20 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53755) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dRFly-0006RO-QK for bug-gnu-emacs@gnu.org; Sat, 01 Jul 2017 06:37:15 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dRFlu-0005Tt-Mv for bug-gnu-emacs@gnu.org; Sat, 01 Jul 2017 06:37:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:44262) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dRFlu-0005Tl-JW for bug-gnu-emacs@gnu.org; Sat, 01 Jul 2017 06:37:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1dRFlu-0003BO-ED for bug-gnu-emacs@gnu.org; Sat, 01 Jul 2017 06:37:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 01 Jul 2017 10:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 27544 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 27544-submit@debbugs.gnu.org id=B27544.149890540712210 (code B ref 27544); Sat, 01 Jul 2017 10:37:02 +0000 Original-Received: (at 27544) by debbugs.gnu.org; 1 Jul 2017 10:36:47 +0000 Original-Received: from localhost ([127.0.0.1]:46939 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dRFlf-0003As-9q for submit@debbugs.gnu.org; Sat, 01 Jul 2017 06:36:47 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:34457) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dRFld-0003Af-ML for 27544@debbugs.gnu.org; Sat, 01 Jul 2017 06:36:45 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dRFlU-0005E2-Ga for 27544@debbugs.gnu.org; Sat, 01 Jul 2017 06:36:40 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:55370) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dRFlU-0005Dw-Cw; Sat, 01 Jul 2017 06:36:36 -0400 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2461 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dRFlT-00039b-Lp; Sat, 01 Jul 2017 06:36:36 -0400 In-reply-to: (message from Itai Berli on Sat, 1 Jul 2017 12:58:28 +0300) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:134067 Archived-At: > From: Itai Berli > Date: Sat, 1 Jul 2017 12:58:28 +0300 > > Emacs supports 12 Unicode bidirectional marks (ALM, RLM, LRM, LRE, > RLE, LRO, RLO, PDF, FSI, LRI, RLI, and PDI), each of which displays as > a very thin space. This raises two problems. > > 1. On the one hand, the fact that these inherently invisible > marks manifest, by default, as thin spaces undermines attempts at > precise alignment and positioning. Moreover, in the case of LRM, RLM > and ALM, this behavior contradicts explicit directions given in the > Unicode > Bidirectional Algorithm 8.0.0 specifications (section 2.6 Implicit > Directional Marks): > > they do not appear in the display > (To my understanding, this is meant to apply to all bidi marks, even > if only stated explicitly for LRM, RLM and ALM.) > > 2. On the other hand, the fact that these spaces are so thin as to be > barely noticeable, and the fact that > they are indistinguishable from one another makes it difficult to debug > and resolve strange and/or erroneous behavior that can happen in a > bidi document, an example of which is given below. The above is the default way these control characters are displayed. This default was chosen so as to, on the one hand avoid making them entirely invisible, as doing that was deemed un-Emacsy, and OTOH make them barely visible, so that they won't disrupt the legibility of the displayed text. However, Emacs being Emacs, this is just the default, and it can be changed. The visual appearance of these (and other similar) characters can be customized via the variable 'glyphless-char-display-control', which is described in the Emacs manual, and in more detail in the ELisp manual. > The solution to both problems is to make the bidi marks visible in > `whitespace` mode only, and to give them glyphs that are (a) easy to > notice, (b) distinguishable from other whitespace visualization glyphs, (c) > distinct from one another. You can do that using 'glyphless-char-display-control'. If that is somehow not enough, you could also define a display-table entry for these characters, specifically for whitespace-mode. Patches to that effect are welcome (I think this should be a user option, if we want such a feature). > If we were able to visualize the whitespace, we would have realized from > the beginning that the sequence of characters in this paragraph was, from > left to right: > > RTL-RTL-RLO-RLO-H-e-l-l-o-PDO-,-PDO-SPACE-w-o-r-l-d-! > > Thus, our first three actions removed the first three characters, leaving us > with: > > RLO-H-e-l-l-o-PDO-,-PDO-SPACE-w-o-r-l-d-! > > We now realize that even the final, correct form, is in fact littered > with bidi errors and potential landmines! Overriding the bidi attributes with the likes of RLO can indeed lead to confusing display. Emacs has functions that Lisp applications can use to discover these confusing situations, where the application would like to warn users. See the description of bidi-find-overridden-directionality in the ELisp manual.