From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: how reliable is rendering of complex scripts? Date: Sun, 04 Oct 2015 10:07:23 +0300 Message-ID: <83zizz1590.fsf@gnu.org> References: <20151002.073900.472512461.wl@gnu.org> <83pp0x7mnq.fsf@gnu.org> <20151004.063952.389316157.wl@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1443942478 7568 80.91.229.3 (4 Oct 2015 07:07:58 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 4 Oct 2015 07:07:58 +0000 (UTC) Cc: handa@gnu.org, emacs-devel@gnu.org To: Werner LEMBERG Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Oct 04 09:07:48 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZidOd-00061F-23 for ged-emacs-devel@m.gmane.org; Sun, 04 Oct 2015 09:07:47 +0200 Original-Received: from localhost ([::1]:41484 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZidOc-0007tf-7A for ged-emacs-devel@m.gmane.org; Sun, 04 Oct 2015 03:07:46 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:37992) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZidOO-0007tE-2u for emacs-devel@gnu.org; Sun, 04 Oct 2015 03:07:33 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZidOK-0006X9-RM for emacs-devel@gnu.org; Sun, 04 Oct 2015 03:07:32 -0400 Original-Received: from mtaout24.012.net.il ([80.179.55.180]:45196) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZidOK-0006Wp-EU; Sun, 04 Oct 2015 03:07:28 -0400 Original-Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0NVO00400OO8P700@mtaout24.012.net.il>; Sun, 04 Oct 2015 10:00:23 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([84.94.185.246]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NVO004U3OSNDZ00@mtaout24.012.net.il>; Sun, 04 Oct 2015 10:00:23 +0300 (IDT) In-reply-to: <20151004.063952.389316157.wl@gnu.org> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 80.179.55.180 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:190851 Archived-At: > Date: Sun, 04 Oct 2015 06:39:52 +0200 (CEST) > Cc: emacs-devel@gnu.org, handa@gnu.org > From: Werner LEMBERG > > >> I wonder how reliable emacs displays complex scripts like > >> Devanagari or Arabic. > > > > AFAIK, no one has ever performed a study about this, let alone > > repeated it when the relevant standards changed. > > This essentially means that Emacs developers wait for users to report > bad renderings, right? No, it means Emacs developers lack resources, knowledge, and motivated individuals aboard to do that. If you or someone else volunteers, that'd be great. Where we do have resources, we are generally ahead of HarfBuzz: e.g., the implementation of UAX#9 compliant with Unicode 6.3 - 8.0 was in Emacs many moons before it was anywhere else in the Free Software world, including HarfBuzz. > >> For example, the maintainers of the HarfBuzz library did extensive > >> comparisons of the rendering results with the MS engine to iron out > >> zillions of small buglets in OpenType handling. > > > > At least on MS-Windows, Emacs uses the MS engine directly, so some > > of similar buglets should not affect us on Windows. > > Well, this makes Emacs on MS-Windows really superior to other > platforms in this area, which is less than ideal... I mean `superior' > in the sense that the rendering results on MS-Windows are well tested > and can be trusted in general, something that is missing otherwise. That's not entirely true, because, as I said, part of the data and algorithms needed for complex script layout is in Emacs, and is used on all supported platforms, not just by the Windows build. > Given that HarfBuzz is very mature today, and that it has been > extensively tested against Windows rendering results, and that it also > contains a large corpus of test cases for complex ligatures together > with a simple test TTY program (`hb-shape'), I suggest that someone > (probably Ken'ichi) writes a similar test program for libm17n so that > diffing would be possible. > > https://github.com/behdad/harfbuzz/tree/master/test/shaping > https://github.com/behdad/harfbuzz/tree/master/util Thanks for the pointer. I will see if we can extract something from there to see how Emacs displays those scripts. > >> are there test suites to compare the results? > > > > There's a test suite for bidirectional display, but it only tests > > the reordering of characters for display, not the shaping. There's > > nothing else, AFAIK. If you, or someone else, can work on adding > > one, that'd be great. > > In case someone is working on this issue, asking the HarfBuzz > developer for assistance might be a good thing. I guess that they > have even larger corpora that could be probably provided for testing > purposes. No one is working on this, to the best of my knowledge. Once again, we lack individuals on board who understand these issues well enough to do that. A single bug in libm17n-flt reported a few months ago, discovered in a script whose support is not even in Emacs, was fixed by Handa-san, and that's about all the activity that I remember.