From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bidi,gmane.emacs.devel Subject: Re: Mixed L2R and R2L paragraphs and horizontal scroll Date: Fri, 12 Feb 2010 13:03:52 +0200 Message-ID: <83ljeyd8br.fsf@gnu.org> References: <83tyu3iu6b.fsf@gnu.org> <83vdeghfqg.fsf@gnu.org> <201002012205.o11M5Sci011809@beta.mvs.co.il> <83k4uvh09o.fsf@gnu.org> <201002031310.o13DAqXd019253@beta.mvs.co.il> <40314.130.55.118.19.1265230948.squirrel@webmail.lanl.gov> <201002041621.o14GL6w5006928@beta.mvs.co.il> <833a1ghjrj.fsf@gnu.org> <83tytwf1tp.fsf@gnu.org> <30fb12601002111340m26c80bcfi69906ac90d887684@mail.gmail.com> Reply-To: Eli Zaretskii NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE X-Trace: ger.gmane.org 1265972938 27517 80.91.229.12 (12 Feb 2010 11:08:58 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 12 Feb 2010 11:08:58 +0000 (UTC) Cc: emacs-bidi@gnu.org, emacs-devel@gnu.org To: Beni Cherniavsky Original-X-From: emacs-bidi-bounces+gnu-emacs-bidi=m.gmane.org@gnu.org Fri Feb 12 12:08:54 2010 Return-path: Envelope-to: gnu-emacs-bidi@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1NftNK-0005L2-Uy for gnu-emacs-bidi@m.gmane.org; Fri, 12 Feb 2010 12:08:53 +0100 Original-Received: from localhost ([127.0.0.1]:49288 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NftNC-0008Dd-Pk for gnu-emacs-bidi@m.gmane.org; Fri, 12 Feb 2010 06:07:46 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NftN5-0008Cn-3P for emacs-bidi@gnu.org; Fri, 12 Feb 2010 06:07:39 -0500 Original-Received: from [140.186.70.92] (port=50753 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NftMy-00086k-4I for emacs-bidi@gnu.org; Fri, 12 Feb 2010 06:07:38 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1NftMb-00021A-5a for emacs-bidi@gnu.org; Fri, 12 Feb 2010 06:07:12 -0500 Original-Received: from mtaout22.012.net.il ([80.179.55.172]:59700) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1NftMa-000214-Oj; Fri, 12 Feb 2010 06:07:09 -0500 Original-Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0KXQ004006Q2YD00@a-mtaout22.012.net.il>; Fri, 12 Feb 2010 13:06:00 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([87.70.67.249]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0KXQ00M506TUVVH0@a-mtaout22.012.net.il>; Fri, 12 Feb 2010 13:05:56 +0200 (IST) In-reply-to: <30fb12601002111340m26c80bcfi69906ac90d887684@mail.gmail.com> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-BeenThere: emacs-bidi@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of Emacs support for multi-directional text." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-bidi-bounces+gnu-emacs-bidi=m.gmane.org@gnu.org Errors-To: emacs-bidi-bounces+gnu-emacs-bidi=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bidi:554 gmane.emacs.devel:121069 Archived-At: > From: Beni Cherniavsky > Date: Thu, 11 Feb 2010 23:40:03 +0200 > Cc: Stefan Monnier , emacs-bidi@gnu.org, = emacs-devel@gnu.org >=20 > [Sorry, long mail. In the first half I'm whining about why I don't > like Eli's solution; but I also reply with technical ideas below...= ] Thank you for your comments. > - Truncation in logical order might(?) be OK if coupled with > logical-order "mirrored" scrolling. What is ``logical-order "mirrored" scrolling''? > Worse yet, if I now proceed to *edit* the buffer, I'll modify it = in > completely wrong places, and even when I realize that, fixing it = will > be even harder! I'll need to *simultaneously* reverse-engineer y= our > deviant bidi algorithm and figure out the real logical order, and > then very carefully fix my edits, all the time getting strangely > permuted feedback for my actions. I don't think fixing such problems is anywhere near that hard. Just display the text in its logical order (a flip of a buffer-local variable) and fix it. Case closed. That doesn't mean that we should proliferate problems that need fixing, of course. > This is the *real* reason we hate broken bidi support. This is not really fair. I'm not developing a ``broken bidi support''. Everything developed so far is first-class bidirectional operation, as much as I know what that means, at least on the low level on which I'm working. Line continuation is the first issue where I gave up on making it 100% perfect, because it's just too damn hard for someone who has only weekends to work on that, and doesn't have enough knowledge and experience in Emacs display-related features. The truth is that this is not a job for a bidi expert, it is a job fo= r an Emacs display engine expert who can ask for bidi advice from time to time. But I've waited for 8 years for such a display engine expert to come and integrate the bidi reordering code I wrote with Emacs redisplay. It never happened. So now I'm doing it as best I can, in the hope that my best will be good enough. People who want the result to be better can help by suggesting very specific implementation ideas, based on specific details of the Emacs redisplay code. General ideas are generally not helpful, because my problem is not with principles, it is with the details. So if you want to help, please make yourself familiar with xdisp.c, which is th= e bulk of the display engine. Then if you have specific ideas about ho= w to implement continuation lines that read in the correct order, I'll be all ears. > No bidi at all > is frequently better - ain't pretty but at least has 1:1 mental m= odel. Maybe I should just quit, then. I don't want my name scribbled on code that is considered worse than what Emacs was before that. I could find better uses for my scarce free time. > To be fair, we're talking about rare situations where embedded text= is > broken across lines. But note that a wrong base direction can infl= ict > this on whole paragraphs (more on that below). Unlike with other programs, in Emacs, wrong base direction is very easy to fix, even with what I have now. There's a per-buffer variabl= e that forces one of the two directions on all the paragraphs in the buffer, and if that's not fine-grained enough, you can insert the corresponding directional mark at the beginning of the paragraph. (Eventually, there should be commands to do this, without asking the user to remember the Unicode codepoints of these characters.) > > =C2=A0. I saw no other editor that supports truncation and behave= s > > =C2=A0 =C2=A0otherwise. =C2=A0(I don't know about any editors tha= t support > > =C2=A0 =C2=A0continuation lines like Emacs does.) =C2=A0See below= . > > > Truncation is OK, but the issue is continuation. >=20 > Not following your claim about editors that support continuation - > all these do and behave otherwise (i.e. as Ehud wants): > Notepad, gedit, firefox/webkit, OpenOffice. No, they don't have continuation lines. They reflow the lines, i.e. divide text differently between the lines. > Indeed, embedded text tends to be short. >=20 > But I'm afraid it's bigger than you think, because if the base dire= ction > of a paragraph is incorrect, *the whole paragraph* will wrap in thi= s > broken bottom-up manner. See above: this is easy to fix in Emacs. Again, that's not a good reason to have incorrect display, of course. > This also means that forcing all paragraphs to R2L or L2R base dire= ction > (which would be a handy way to momentarily work around wrong imperf= ect > guessing) would break line order in half the paragraphs in a mixed = buffer! Let's not get carried away: the problem will only be with continued lines, not with every line. > So if only the line breaking points were static, you'd have no > performance problem! If the line breaking points were at known buffer positions, yes. (I don't quite know what you mean by ``static''.) > =3D> Could you maybe cache this information and recompute it only w= hen > the line is edited? This is unlikely to help, with the current design of the display engine. It works very hard to avoid redisplaying lines that did not change. So when it finally decides to redisplay a line, there are very good chances that the cached value is invalid anyway. Note that even placing a text property on some text of a line could make it overflow the display margin at a different point: for example, giving the text bold or italics face is all you need for the continuation point to move. > [XEmacs already has a "Line Start Cache" according to its Internals= Manual. So does Emacs. But the cache is invalidated when/where the text changes. > I didn't find a similar overview for Emacs. Is there anything I ca= n read > to understand Emacs redisplay before I attempt to approach the sour= ce?] Only the comments at the beginning of xdisp.c. After that, read the code, guided by the high-level structure described in those comments. That, asking questions here, and related discussions on this list is all you have, sorry. > > until we discover where we should stop. =C2=A0(We could do a bina= ry search, > > of course, but that's details.) =C2=A0I don't think that's reason= able, and > > I have no idea what will this do to the redisplay speed. > > > Binary search is a big improvement! In 10 attempts you can handle = lines > of 1K chars, in 20 - 1M. On my computer Emacs presently handles 10= 0k > smoothly, 1M already feels sluggish. By crude (and probably wrong) > computation, binary search would still be fast enough up to 10K... The problem is not with implementing binary search, the problem is to plug it into the display engine, which needs to be aware of all the possible side effects of each thing it does while preparing display o= f a line. > Also, I presume that the heavy part of a redisplay is normally the = actual > output to screen (if not, why do such a complex job minimizing it?)= . I don't think so. I think the heavy part is computing and merging faces. But doing measurements to find out the hot spots would be in itself a useful project, which may help down the line. Volunteers ar= e welcome. > To top this, I think you can do several times better if you allow s= ome > imprecision in line breaking of mixed-direction paragraphs. Natura= lly, > you must not overshoot the screen, but some undershooting is OK. S= o it > seems to me that you could reasonably do it with a greedy approach: >=20 > (1) Add characters in *logical order* as long as they fit. > (2) Try it in visual order to account for precise typographic stuff= . > (3) As long as it doesn't fit, strip one a char and retry (2). > (4) When OK, repeat with actual output display to the screen. I think this will only work if there are no embeddings. In text with embeddings, doing it in logical order will have you scan wrong characters anyway, and you are back at square one. > Finally, I want to propose a feature that I think will be handy, > and also happens to support efficient wrapping. The truth is that = any > way to wrap an embedding accross lines is ugly! I'd like a mode wh= ere > any embedding either fits completely on a line or starts and ends o= n a > lines by itself: >=20 > +----------------------------------------+ > |some latin text followed by \| > |\ ROF TXET GNOL TAHWEMOS WERBEH| > |\ SIHT GNITARTSNOMED| > |followed by latin tail | > +----------------------------------------+ Again, what about embeddings? I want to have full UAX#9 support, not just some simplified bidi that breaks as soon as the user wants to us= e the full power of the Unicode algorithm.