From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bidi,gmane.emacs.devel Subject: Re: Merging the bidi branch Date: Tue, 30 Mar 2010 12:33:46 +0300 Message-ID: <83wrwuywzp.fsf@gnu.org> References: <83tyrz1gfr.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: lo.gmane.org X-Trace: dough.gmane.org 1269941650 32386 80.91.229.12 (30 Mar 2010 09:34:10 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 30 Mar 2010 09:34:10 +0000 (UTC) Cc: emacs-bidi@gnu.org To: emacs-devel@gnu.org Original-X-From: emacs-bidi-bounces+gnu-emacs-bidi=m.gmane.org@gnu.org Tue Mar 30 11:34:03 2010 Return-path: Envelope-to: gnu-emacs-bidi@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1NwXpf-0005Gk-C1 for gnu-emacs-bidi@m.gmane.org; Tue, 30 Mar 2010 11:33:59 +0200 Original-Received: from localhost ([127.0.0.1]:49117 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NwXpe-0006bt-Jw for gnu-emacs-bidi@m.gmane.org; Tue, 30 Mar 2010 05:33:58 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NwXpT-0006YZ-Az for emacs-bidi@gnu.org; Tue, 30 Mar 2010 05:33:47 -0400 Original-Received: from [140.186.70.92] (port=36494 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NwXpP-0006Wz-8A for emacs-bidi@gnu.org; Tue, 30 Mar 2010 05:33:46 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1NwXpN-0003jN-Ku for emacs-bidi@gnu.org; Tue, 30 Mar 2010 05:33:43 -0400 Original-Received: from mtaout20.012.net.il ([80.179.55.166]:52264) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1NwXpN-0003j1-DQ; Tue, 30 Mar 2010 05:33:41 -0400 Original-Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0L0300C00962VI00@a-mtaout20.012.net.il>; Tue, 30 Mar 2010 12:33:39 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([77.127.176.135]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0L03009KG982Y570@a-mtaout20.012.net.il>; Tue, 30 Mar 2010 12:33:39 +0300 (IDT) In-reply-to: X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-BeenThere: emacs-bidi@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of Emacs support for multi-directional text." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-bidi-bounces+gnu-emacs-bidi=m.gmane.org@gnu.org Errors-To: emacs-bidi-bounces+gnu-emacs-bidi=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bidi:556 gmane.emacs.devel:122901 Archived-At: > From: Stefan Monnier > Cc: emacs-devel@gnu.org > Date: Mon, 29 Mar 2010 10:41:33 -0400 > > > I'm getting ready for the merge of the bidi branch with the trunk. It > > built successfully on MS-Windows and passed some simple testing in a > > GUI session. (Yes, I promised this only in 2 weeks time; I lied.) So > > if everything goes according to plan, I will merge before noon > > tomorrow (GMT+3). > > Yay! It's done. However, given the excitement that this feature generates, I would like to lower everybody's expectations somewhat, to avoid disappointment and frustration. The support for bidirectional editing, as it was merged into the current trunk is still very much experimental. I do not recommend turning it on for any purpose but hacking or debugging it. The chance of losing your work or crashing Emacs while working in a bidi-reordered buffer is quite high. Basic Emacs features has not yet been made to work reliably in such buffers. Vertical cursor motion is a prominent example: it mostly works, but is known to infloop or crash under some circumstances (because the underlying primitives use `current_column' and `move_to_column' which assume unidirectional display). Use C-f, C-b, M-<, and M-> instead: they do work. Faces, region highlighting, and invisible text work, but `display' properties with string values do not. Display of images was not yet tested, and in any case, images will probably not be reordered correctly wrt surrounding bidirectional text, with the current code. Composed characters will probably cause messed up display (try "C-h H" and see for yourself). Etc., etc. YOU HAVE BEEN WARNED! That said, if you are still un-intimidated, read the entry in NEWS and the section in the manual to which it points, and start using and hacking this new feature. I need help in developing this further. The amount of work required to bring this to the level where it can be released is enormous. The good news are that a large portion of the job does not need any knowledge of bidirectional scripts or UAX#9. Here are a few tasks that are quite high on my todo and where I could use help: . Display of right-to-left paragraphs. This currently works correctly only on a TTY. I need help from experts on GUI back-ends (X, MS-Windows, and NS) to make it work in GUI sessions. Each glyph_row structure that needs to be displayed starting at the right margin has a special flag reversed_p turned on, and its glyphs are already in the reversed order. The back-end needs to implement two things: o when this flag is on, draw the glyphs flushed all the way to the right margin of the window, and o if the row width is less than the window's, extend the default face to the left margin of the window . The bidi reordering code in src/bidi.c uses a table of bidirectional properties of characters. The current code constructs this table from static data that was gleaned from the Unicode database, but needs to be manually fixed each time Unicode database changes. This needs to be rewritten to use data in uni-bidi.el. The challenge here is to use a memory-efficient data structure and (if the data turns out to be significant) decide when to load that data. . Similarly for the bidi-mirrored properties of characters: we need to use data from the Unicode database through uni-*.el files. The current implementation uses a small table that only supports ASCII characters, which is clearly insufficient. . Turn on features needed for shaping bidirectional scripts. This is needed for Arabic ligatures, at the very least. People who are familiar with libotf and Uniscribe shaping engines used by Emacs are welcome to work on this. I myself have exactly zero knowledge about these issues. There's some potential difficulty here: these shaping engines normally reorder characters for display themselves; we need to find a way of getting them to perform the shaping without reordering. . Display of composite characters. In addition to making it bidi-aware, we need to figure out how to display characters with diacriticals, because the current bidi reordering code reorders them as well, placing the diacritical before the character.