From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: handa@gnu.org (K. Handa) Newsgroups: gmane.emacs.bugs Subject: bug#20140: 24.4; M17n shaper output rejected Date: Wed, 25 Mar 2015 23:25:54 +0900 Message-ID: <87mw31887h.fsf@gnu.org> References: <20150318222040.4066e6e9@JRWUBU2> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1427293601 1046 80.91.229.3 (25 Mar 2015 14:26:41 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 25 Mar 2015 14:26:41 +0000 (UTC) Cc: 20140@debbugs.gnu.org To: Richard Wordingham Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Mar 25 15:26:30 2015 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YamG8-0004Nc-Et for geb-bug-gnu-emacs@m.gmane.org; Wed, 25 Mar 2015 15:26:17 +0100 Original-Received: from localhost ([::1]:39373 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YamG7-0002di-MQ for geb-bug-gnu-emacs@m.gmane.org; Wed, 25 Mar 2015 10:26:15 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:54137) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YamFz-0002VD-Vr for bug-gnu-emacs@gnu.org; Wed, 25 Mar 2015 10:26:12 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YamFv-000340-Cf for bug-gnu-emacs@gnu.org; Wed, 25 Mar 2015 10:26:07 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:46710) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YamFv-00033l-8z for bug-gnu-emacs@gnu.org; Wed, 25 Mar 2015 10:26:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1YamFu-0008KS-O6 for bug-gnu-emacs@gnu.org; Wed, 25 Mar 2015 10:26:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: handa@gnu.org (K. Handa) Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 25 Mar 2015 14:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20140 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 20140-submit@debbugs.gnu.org id=B20140.142729356032005 (code B ref 20140); Wed, 25 Mar 2015 14:26:02 +0000 Original-Received: (at 20140) by debbugs.gnu.org; 25 Mar 2015 14:26:00 +0000 Original-Received: from localhost ([127.0.0.1]:36486 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YamFr-0008K8-8R for submit@debbugs.gnu.org; Wed, 25 Mar 2015 10:25:59 -0400 Original-Received: from fencepost.gnu.org ([208.118.235.10]:54581 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YamFo-0008Jy-1J for 20140@debbugs.gnu.org; Wed, 25 Mar 2015 10:25:57 -0400 Original-Received: from fl1-122-134-88-3.iba.mesh.ad.jp ([122.134.88.3]:56196 helo=tinhau) by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1YamFn-0007fb-8G; Wed, 25 Mar 2015 10:25:55 -0400 Original-Received: from handa by tinhau with local (Exim 4.80) (envelope-from ) id 1YamFm-0004U5-Jj; Wed, 25 Mar 2015 23:25:54 +0900 In-Reply-To: <20150321175818.1b125eba@JRWUBU2> (message from Richard Wordingham on Sat, 21 Mar 2015 17:58:18 +0000) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:100923 Archived-At: Hi, thank you for the detailed explanation. In article <20150321175818.1b125eba@JRWUBU2>, Richard Wordingham writes: > What I ought to want is SIL's split cursor scheme, which indicated the > next ('point') and previous characters, even in bidirectional text. > Unfortunately, that's not compatible with m17n, which seems to assume > that cursor position will be a single number. The Emacs functions > forward-char-intrusive and backward-char-intrusive provided a pleasant, > more intuitive, alternative, and I am sad to hear they are gone. > Perhaps I'll have to start using toggle-auto-composition. Those Emacs functions are just my idea for improving Emacs for CTL users, and have never been included in the official Emacs verison. I check the code and found two problems: (1) When the command sets disable-point-adjustment to t, command_loop_1 should force updating the display if point is within a grapheme cluster. So we need this patch: diff --git a/src/keyboard.c b/src/keyboard.c index bf65df1..13125c1 100644 --- a/src/keyboard.c +++ b/src/keyboard.c @@ -1636,6 +1636,16 @@ command_loop_1 (void) adjust_point_for_property (last_point_position, MODIFF !=3D prev_modiff); } + else if (current_buffer =3D=3D prev_buffer + && last_point_position !=3D PT) + { + if (PT > BEGV && PT < ZV + && (composition_adjust_point (last_point_position, PT) !=3D PT)) + /* Now point is within a grapheme cluster. We must update + the display so that this cluster is discomosed on the + screen and the cursor is correctly placed at point. */ + windows_or_buffers_changed =3D 22; + } =20 /* Install chars successfully executed in kbd macro. */ =20 (2) We should break a grapheme cluster at point. So we need this patch. diff --git a/src/xdisp.c b/src/xdisp.c index a17f5a9..0c56395 100644 --- a/src/xdisp.c +++ b/src/xdisp.c @@ -3408,6 +3408,9 @@ compute_stop_pos (struct it *it) pos =3D next_overlay_change (charpos); if (pos < it->stop_charpos) it->stop_charpos =3D pos; + /* If point is in front of the current stop pos, stop there. */ + if (charpos < PT && PT < it->stop_charpos) + it->stop_charpos =3D PT; =20 /* Set up variables for computing the stop position from text property changes. */ @@ -8166,7 +8169,12 @@ next_element_from_buffer (struct it *it) && IT_CHARPOS (*it) >=3D it->redisplay_end_trigger_charpos) run_redisplay_end_trigger_hook (it); =20 - stop =3D it->bidi_it.scan_dir < 0 ? -1 : it->end_charpos; + /* Set stop position considering the bidi direction and point. */ + if (it->bidi_it.scan_dir < 0) + stop =3D (PT < IT_CHARPOS (*it)) ? PT : -1; + else + stop =3D ((IT_CHARPOS (*it) < PT && PT < it->end_charpos) + ? PT : it->end_charpos); if (CHAR_COMPOSED_P (it, IT_CHARPOS (*it), IT_BYTEPOS (*it), stop) && next_element_from_composition (it)) Could you try these patches and test the usability of forward-char-intrusive and backward-char-intrusive? > > Please try to move cursor over this Devanagri text "=E0=A4=B9=E0=A4=BF= =E0=A4=82=E0=A4=A6=E0=A5=80" on > > Emacs, gedit, and, for instance, firefox. They all treat > > that text as 2 grapheme clusters "=E0=A4=B9=E0=A4=BF=E0=A4=82" and "=E0= =A4=A6=E0=A5=80". The first > > one corresponds to character the sequence U+935 U+93F, and > > U+93F (vowel I) is displayed before U+935 (base cosonant). > Note that those clusters are only 3 and 2 characters long. Retyping > them is tolerable. Now consider the Sanskrit Devanagari text =E0=A4=B8= =E0=A5=8D=E0=A4=A4=E0=A5=8D=E0=A4=B0=E0=A5=80, > which contains two consonant-combining viramas. Emacs moves across it > in 1 step, but Claws e-mail (GTK-based, I believe) and LibreOffice > (HarfBuzz-based, at least for linux) both take 3 steps to move across > it. Claws and LibreOffice use different algorithms to position the > cursor. That of LibreOffice seems more reasonable, but that of > Claws works better! The reason is that Unicode did not declare virama > as forming grapheme clusters. Ah, hmmm, that a problem of DEVA-OTF.flt and DEV2-OTF.flt of the m17n library. I'll try to fix them. > It seems to have solved all of them. When I reported the bug, I was > having problems with my font because libotf was silently ignoring half > the lookups in my font. Could you please send me (not on this list) an appropriate bug/problem report if libotf should be fixed? > I though I might have problems with U+1A58 TAI THAM SIGN MAI KANG LAI, > which in Lao visually groups (usually) with the following base > consonant and in Tai Khuen groups with the preceding base consonant. My > clustering in Emacs follows the Tai Khuen scheme. (I compose two > orthographic clusters together in Emacs, but declare two grapheme > clusters in the FLT processing.) However, my font follows a major > Northern Thai dictionary and places it on the following base consonant > if there is nothing above it, but otherwise places it on the preceding > base consonant. However, my implementation is too dirty to cause > problems - the second cluster is not reported as deriving from the > mai kang lai character. > I wonder, though, what will happen if I manage to implement the > Universal Shaping Engine's (USE) rphf feature. The author of a Lao-style > Tai Tham font wanted this feature in HarfBuzz. The desired effect seems > easy to achieve in m17n-flt, but placing it under font control is more > difficult. I'm studying MLM2-OTF.flt to see how to do it. I've just started to study the Universal Shaping Engine. It seems that we can implement it by a proper FLT file. > > > However, it then makes editing of the 'clusters' more > > > difficult. Note that there are examples above with 5 > > > characters in a cluster, and this is by no means the > > > limit. > >=20 > > But, it seems that the current behavior is accepted, at > > least, by Indic people. > Who do you mean by 'Indic people'? I just mean that I have not heard any complaints about that "too long cluster problem" of Emacs. No one is using Emacs for Indic scripts? > New Tai Lue is an interesting case. Microsoft delayed support for this > simple Indic script for so long that most apparently Unicode-encoded > New Tai Lue text was actually encoded in visual order. With Unicode > 8.0, New Tai Lue is changing from phonetic order to visual order, and > it will no longer need any clusters at all!=20=20 Wow, I didn't know that. > Emacs 23.3 (which is what is in long-term support Ubuntu > 12.04) offers no support for New Tai Lue, so I am not sure > that there is yet a New Tai Lue view on composition in > Emacs. We may be able to provide supports for new scripts in elpa. --- K. Handa handa@gnu.org