From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n) Date: Sat, 29 Dec 2018 16:49:23 +0200 Message-ID: <838t085qx8.fsf@gnu.org> References: <20181222151509.GC2244@macbook.localdomain> <83h8f5a7po.fsf@gnu.org> <20181222154945.GE2244@macbook.localdomain> <83bm5d9wsc.fsf@gnu.org> <20181222205948.GF2244@macbook.localdomain> <838t0gapcj.fsf@gnu.org> <20181223135109.GA6568@macbook.localdomain> <83va3k8c79.fsf@gnu.org> <20181224020847.GC6568@macbook.localdomain> <83lg4e9a7q.fsf@gnu.org> <20181224173723.GH6568@macbook.localdomain> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1546094892 15617 195.159.176.226 (29 Dec 2018 14:48:12 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 29 Dec 2018 14:48:12 +0000 (UTC) Cc: behdad@behdad.org, far.nasiri.m@gmail.com, 33729@debbugs.gnu.org To: Khaled Hosny Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Dec 29 15:48:07 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gdFuJ-0003xD-IK for geb-bug-gnu-emacs@m.gmane.org; Sat, 29 Dec 2018 15:48:07 +0100 Original-Received: from localhost ([127.0.0.1]:38737 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gdFwQ-00072p-Eo for geb-bug-gnu-emacs@m.gmane.org; Sat, 29 Dec 2018 09:50:18 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:49381) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gdFwE-00072O-7P for bug-gnu-emacs@gnu.org; Sat, 29 Dec 2018 09:50:12 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gdFwA-0008Lr-EZ for bug-gnu-emacs@gnu.org; Sat, 29 Dec 2018 09:50:06 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:55379) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gdFwA-0008Lb-9Q for bug-gnu-emacs@gnu.org; Sat, 29 Dec 2018 09:50:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gdFwA-0000yd-0u for bug-gnu-emacs@gnu.org; Sat, 29 Dec 2018 09:50:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 29 Dec 2018 14:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33729 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 33729-submit@debbugs.gnu.org id=B33729.15460949923734 (code B ref 33729); Sat, 29 Dec 2018 14:50:01 +0000 Original-Received: (at 33729) by debbugs.gnu.org; 29 Dec 2018 14:49:52 +0000 Original-Received: from localhost ([127.0.0.1]:40940 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gdFw0-0000yA-Cq for submit@debbugs.gnu.org; Sat, 29 Dec 2018 09:49:52 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:34081) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gdFvz-0000xw-9G for 33729@debbugs.gnu.org; Sat, 29 Dec 2018 09:49:51 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gdFvq-00085v-A9 for 33729@debbugs.gnu.org; Sat, 29 Dec 2018 09:49:45 -0500 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48587) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gdFvq-00085i-55; Sat, 29 Dec 2018 09:49:42 -0500 Original-Received: from [176.228.60.248] (port=3863 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gdFvp-0002Gl-PR; Sat, 29 Dec 2018 09:49:42 -0500 In-reply-to: <20181224173723.GH6568@macbook.localdomain> (message from Khaled Hosny on Mon, 24 Dec 2018 19:37:23 +0200) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:153995 Archived-At: > Date: Mon, 24 Dec 2018 19:37:23 +0200 > From: Khaled Hosny > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > On Mon, Dec 24, 2018 at 06:10:49PM +0200, Eli Zaretskii wrote: > > > Date: Mon, 24 Dec 2018 04:08:47 +0200 > > > From: Khaled Hosny > > > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org, > > > 33729@debbugs.gnu.org, kaushal.modi@gmail.com > > > > > > I think we are almost good now. There is only one serious FIXME left: > > > > > > /* FIXME: guess_segment_properties is BAD BAD BAD. > > > * we need to get these properties with the LGSTRING. */ > > > #if 1 > > > hb_buffer_guess_segment_properties (hb_buffer); > > > #else > > > hb_buffer_set_direction (hb_buffer, XXX); > > > hb_buffer_set_script (hb_buffer, XXX); > > > hb_buffer_set_language (hb_buffer, XXX); > > > #endif > > > > > > We need to know, for a given lgstring we are shaping: > > > * Its direction (from applying bidi algorithm). Each lgstring we are > > > shaping must be of a single direction. > > > > Communicating this to ftfont_shape_by_hb will need changes in a couple > > of interfaces (the existing shaping engines didn't need this > > information). I will work on this soon. > > Great. Done. Please test. I made sure it compiles, but I couldn't actually test the results, as I don't have access to a GNU/Linux system with GUI display. So it could be that I misunderstood the Harfbuzz APIs, as I was essentially flying blind, guided only by the Harfbuzz docs. In particularly, I hope I understood correctly the way we should leave to Harfbuzz guess the properties not explicitly provided by the Emacs context, both for the direction of the text and its script. > > > * Its script, possibly after applying something like: > > > http://unicode.org/reports/tr24/#Common > > > > Per previous discussions, we decided to use the Harfbuzz built-in > > methods for determining the script, since Emacs doesn't have this > > information, and adding it will just do the same as Harfbuzz does, > > i.e. find the first character whose script is not Common etc., using > > the UCD database. I think it was you who suggested to use the > > Harfbuzz built-ins in this case. > > The built-in HarfBuzz code is for getting the script for a given > character, but resolving characters with Common script is left to the > client. Suppose you have this string (upper case for RTL) ABC 123 DEF, > what HarfBuzz sees during shaping is three separate chunks of text ABC, > 123, DEF. The 123 part is all Common script characters and thus > hb_buffer_guess_segment_properties won’t be able to guess anything (and > based on the font and the script, this can cause rendering differences). > Emacs will have to resolve the script of Common characters before > applying bidi algorithm and pass that down to HarfBuzz. See my followup questions about this. For now, I left this aspect to HarfBuzz. > > > * Its language, is Emacs allows setting text language (my understand is > > > that it doesn’t). Some languages really need this for applying > > > language-specfic features (Urdu digits, Serbian alternate glyphs, etc.). > > > > We don't currently have a language property for chunks of text, we > > only have the current global language setting determined from the > > locale (and there's a command to change that for Emacs, should the > > user want it). This is not really appropriate for multilingual > > buffers, but we will have to use that for now, and hope that in the > > future, infrastructure will be added to allow more flexible > > determination of the language of each run of text. (I see that > > Harfbuzz already looks a the locale for its default language, but > > since Emacs allows user control of this, however unlikely, I think > > it's best to use the value Emacs uses.) I will work on this as well. > > Yes, better pass that from Emacs to HarfBuzz. Done, but please see the FIXME I left behind. For testing purposes, you can change the current language like this: M-x set-locale-environment RET xx_YY.CODESET RET For example: M-x set-locale-environment RET sr_RS.UTF-8 RET for the Cyrillic Serbian locale. This should change the value of current-iso639-language to the symbol 'sr'. Please tell if you encounter any difficulties with the code I added, or if you need any further help. Thanks.