From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Werner LEMBERG Newsgroups: gmane.emacs.devel Subject: Re: how reliable is rendering of complex scripts? Date: Sun, 04 Oct 2015 06:39:52 +0200 (CEST) Message-ID: <20151004.063952.389316157.wl@gnu.org> References: <20151002.073900.472512461.wl@gnu.org> <83pp0x7mnq.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1443933629 26192 80.91.229.3 (4 Oct 2015 04:40:29 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 4 Oct 2015 04:40:29 +0000 (UTC) Cc: handa@gnu.org, emacs-devel@gnu.org To: eliz@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Oct 04 06:40:20 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Zib5t-0000pq-MI for ged-emacs-devel@m.gmane.org; Sun, 04 Oct 2015 06:40:17 +0200 Original-Received: from localhost ([::1]:41146 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zib5s-0005qD-Nf for ged-emacs-devel@m.gmane.org; Sun, 04 Oct 2015 00:40:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48237) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zib5f-0005pz-OO for emacs-devel@gnu.org; Sun, 04 Oct 2015 00:40:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zib5e-0000QK-Fq for emacs-devel@gnu.org; Sun, 04 Oct 2015 00:40:03 -0400 Original-Received: from mout.gmx.net ([212.227.15.19]:56142) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zib5a-0000Jx-Kz; Sun, 04 Oct 2015 00:39:58 -0400 Original-Received: from gnu.org ([77.119.130.47]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0LnfuZ-1aHO4I0sz1-00hswI; Sun, 04 Oct 2015 06:39:57 +0200 In-Reply-To: <83pp0x7mnq.fsf@gnu.org> X-Mailer: Mew version 6.7rc2 on Emacs 25.0.50.1 / Mule 6.0 (HANACHIRUSATO) X-Provags-ID: V03:K0:HoviqODVTJzj/7ohNCP9Rmw8wmlnDK6/3PoFLEb9PNUUz+wWrBt fciTHwhzZDAgzBGIeCVBPLRXkMdpGHUezePNiu0xsVgM+gPljYxSgMNlVTc3mKP4eH0no1z DsxAgNaWGaXVF30sZUTWBtHY245NqQwv3Zp520WEBWjmokaHHsX26qZtmOdZ3kMKzxeHvra PlnZXH54jAt9oV8RYyaVg== X-UI-Out-Filterresults: notjunk:1;V01:K0:OcbrvXkO7tw=:lMPXJKbhBqyEabMpV62m65 0RLCZAM0QS8HURMlddtURXnJ8C45uun6JYoK8ivUYfaJuwaFZzylOAcmAVJfDtE+pApTKl4Ut McEsQJm+bgExK3/P9ByKMlqbhakdfnE3gq7PLxgtonoyxAxOnZakpo5PhzYMAr/XCCsC69sc7 CP3G/wdfIVhfnY0xkI/x2u4wFYcuIKsq3fseKBtX5OXg48jgiXG5QTxdQK/7EZiE8Fsq5sCXF FO5fLm8zNbWQAvOryXX8cjNu9hw4/XFPEFLfdsXmcPOWQ8C7/Ukm2JfvFOg89b3TQnBrbWZBl yGAiBK40X9GldM6F3a4aRQk3FhyyAhnjmVKPQ7QAUqZTTLI3JFuM2ZIQWilFMq4VBjYhe1Wt5 1b/fhvet4Vc3GTAh9hVkQUfDCp5r2roMYYvcpEokjmBJTGn2zfe11mag2kfGXF5mOg7uRq49w RHrothqOcJjYhkBqs9l/fBtQuZth34ofU6hT0Svd4ehe6EaC/OZ+TJCc5d6altnbcaDCWoLuP svtJct02fQNUyp4+qtGEroSvet2uVxoiCuDp6Kz98SDR7riJDKXkLMKgoejtTdEtXCmQLfkvW LKEbaxD8xIW5ix8U8Hu5FzrU5X/G29NVkYHyqK/RmPd3YnQ8iahOzMnVl2UP4YIg27hl5n0GK Y6fwY5eH9ReCFSdTrTpayJ4wzZVL1wbpIomHp21WqX4Ufvhfs2FdAi/UKhNQYRgDEunBPF5G4 0TBCwz7Md+co4ujX50e0WeIZxSY7xvHpVsc0WBL9DKGpccpSIqH2cecn8yZQmpH1pElsb6oJ X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.227.15.19 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:190842 Archived-At: >> I wonder how reliable emacs displays complex scripts like >> Devanagari or Arabic. > > AFAIK, no one has ever performed a study about this, let alone > repeated it when the relevant standards changed. This essentially means that Emacs developers wait for users to report bad renderings, right? >> For example, the maintainers of the HarfBuzz library did extensive >> comparisons of the rendering results with the MS engine to iron out >> zillions of small buglets in OpenType handling. > > At least on MS-Windows, Emacs uses the MS engine directly, so some > of similar buglets should not affect us on Windows. Well, this makes Emacs on MS-Windows really superior to other platforms in this area, which is less than ideal... I mean `superior' in the sense that the rendering results on MS-Windows are well tested and can be trusted in general, something that is missing otherwise. >> AFAIK, Emacs relies on the m17n libraries, at least on GNU/Linux (no >> idea about other environments), controlling the OpenType handling >> (partially?) with Lisp code > > That is correct. And while the shaping engines, like libm17n-flt > and Uniscribe, are beyond the scope of Emacs maintenance, the > supporting Lisp and C code is on our table. However, we currently > lack a maintainer in that area (have been lacking for a long time), > so I guess we are not up to speed with the latest developments. I'm > talking first and foremost about the definitions of > character-composition patterns, which tell Emacs which sequences of > characters should be rendered as a single grapheme cluster. There's > a lot to do in this area for various languages. Given that HarfBuzz is very mature today, and that it has been extensively tested against Windows rendering results, and that it also contains a large corpus of test cases for complex ligatures together with a simple test TTY program (`hb-shape'), I suggest that someone (probably Ken'ichi) writes a similar test program for libm17n so that diffing would be possible. https://github.com/behdad/harfbuzz/tree/master/test/shaping https://github.com/behdad/harfbuzz/tree/master/util >> are there test suites to compare the results? > > There's a test suite for bidirectional display, but it only tests > the reordering of characters for display, not the shaping. There's > nothing else, AFAIK. If you, or someone else, can work on adding > one, that'd be great. In case someone is working on this issue, asking the HarfBuzz developer for assistance might be a good thing. I guess that they have even larger corpora that could be probably provided for testing purposes. Werner