From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Pip Cet Newsgroups: gmane.emacs.devel Subject: Re: Ligatures (was: Unify the Platforms: Cairo+FreeType+Harfbuzz Everywhere (except TTY)) Date: Sat, 23 May 2020 21:29:32 +0000 Message-ID: References: <20200517165953.000044d2@web.de> <83lflqblp0.fsf@gnu.org> <83ftbybio3.fsf@gnu.org> <83zha69xs2.fsf@gnu.org> <83367x9qeq.fsf@gnu.org> <0ccae2a4-533b-d15c-2884-c2f00b067776@gmail.com> <83wo5987mk.fsf@gnu.org> <99d4beed-88ae-b5cd-3ecb-a44325c8a1dc@gmail.com> <20200518215908.GA57594@breton.holly.idiocy.org> <83mu6481v3.fsf@gnu.org> <75a90563-51b4-d3b8-4832-fc0e2542af0d@gmail.com> <83blmi7hys.fsf@gnu.org> <837dx55qff.fsf@gnu.org> <834ks95cmz.fsf@gnu.org> <4faa291f-f2df-36d1-73d5-332b93a9b6d8@gmail.com> <83wo544hx5.fsf@gnu.org> <831rnc43ih.fsf@gnu.org> <83ftbs2jr5.fsf@gnu.org> <83lflj16jn.fsf@gnu.org> <83eerb145r.fsf@gnu.org> <831rnb0zld.fsf@gnu.org> <83mu5yzquj.fsf@gnu.org> <831rnazheg.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="11175"; mail-complaints-to="usenet@ciao.gmane.io" Cc: cpitclaudel@gmail.com, alan@idiocy.org, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat May 23 23:31:02 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jcbjQ-0002oM-Ob for ged-emacs-devel@m.gmane-mx.org; Sat, 23 May 2020 23:31:00 +0200 Original-Received: from localhost ([::1]:59920 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcbjP-0002ud-Rn for ged-emacs-devel@m.gmane-mx.org; Sat, 23 May 2020 17:30:59 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:35996) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcbif-0002MF-12 for emacs-devel@gnu.org; Sat, 23 May 2020 17:30:13 -0400 Original-Received: from mail-oo1-xc43.google.com ([2607:f8b0:4864:20::c43]:39222) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jcbid-0006dT-5d; Sat, 23 May 2020 17:30:12 -0400 Original-Received: by mail-oo1-xc43.google.com with SMTP id c194so889455oob.6; Sat, 23 May 2020 14:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Y7amfee69W26TGd5nFvWAoqeUNnVKtZ5rzNvAi0OBYU=; b=YePIuyqwwwwt8sPBXKyElAWuNMxHicjq2Odi3bJjKxuFkuUVn/OAXcbqCbgKkXJ6P5 IYCIagd7aiuJ9INLj+7Nx3PKbM1JsCNjbmOsM1KiVWYDGyTIqJPPJJ7T8TUPC1XbqDWd wM9ycGQLBIcGFFsjQc63NTMj4HZpD9EVaQTJDE28OftXhSQhuvb1xEFBqO+Qk0fmqfFb e/VVdOwcNHulQTs11XSGDL+8zt4MLgPQDaKaGZ7EOTxOhmTpMzEoZo+jtb33OXIjB058 CTP4TLPXPxn2d1tPCcRySd/xSIzKoZzc0cDqvrHaAUZmV9V2dAdSdjDGq/TFysyBkETf 4tvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Y7amfee69W26TGd5nFvWAoqeUNnVKtZ5rzNvAi0OBYU=; b=P6PoDHZE3HIve57kYApcfwg0ihFFZCkcoYdUbzQQSaeN01/dqbpNPvBsFcjNNYL9li HF418s9mBM5GrfFeQ1ecu56xXQmEpPpVsmlweEdpu7e1SMXU6BPuvIEBPrruj5CSVvJD NmMoL0S4LUrPx3gIaQGpVjdXN2jmKgMFmethaHzLppE+9DNrv5Z1CdhABrDbyRJOi/7K fXnsa7DEv49h5Y4A/pI6XYAVReT6hIorwpMWgn/OA2SZHf1mJzN6x8WdWvL7n7VBbS5F m4Ae2/oawaUeVC1sFKjHaC+NAUHP0uIUshsWziWA5XZTo8QK3iZxL3riMJIbhdKmLoBa sNnw== X-Gm-Message-State: AOAM5306iVOweO8mHIfFhybrdQjdgbimH6sPw3wQmuOrI3uGPhdyPeZ8 ihWUzbSvzGIz8IvizMEogDsDCyDS9irULOLMWNnf0R3F X-Google-Smtp-Source: ABdhPJzfs6avFA8DLcRvahNVjl7UnzqbhsOCyMYa4/NjbmW4nSnprURdn+nVCiw4wgl5ljXRnIZfr359C6YWmUvTck4= X-Received: by 2002:a4a:e702:: with SMTP id y2mr8201823oou.44.1590269409145; Sat, 23 May 2020 14:30:09 -0700 (PDT) In-Reply-To: <831rnazheg.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::c43; envelope-from=pipcet@gmail.com; helo=mail-oo1-xc43.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLY=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:251291 Archived-At: On Sat, May 23, 2020 at 5:32 PM Eli Zaretskii wrote: > > From: Pip Cet > > Date: Sat, 23 May 2020 15:13:38 +0000 > > Cc: cpitclaudel@gmail.com, alan@idiocy.org, emacs-devel@gnu.org > > > > As I said, the problem I have is to get look-ahead working, which you > > think isn't a problem. I've got an idea for it, but it doesn't work > > (yet); my theory is the bidi.c code fails to keep its state in the > > iterator and can't deal with multiple parallel iterators. > > > > > I could be wrong, though, so I'm looking > > > forward to see you present a series of patches that do support the > > > existing use cases and the ligatures as well, and don't cause any > > > slowdown in redisplay. > > > > As I said, what's stopping me is the look-ahead problem, and in > > particular some code in bidi.c that doesn't play along well with > > look-ahead. > > I don't think you understand the depth of the issue. I think I do, actually. It's just that you'd prefer the display engine to be torn out by the roots and rewritten in one fell swoop, but that option isn't currently on the table. > If we are going > to send large chunks of text to the shaping engine, then none of the > insane complexity of bidi.c makes sense; we should simply throw all of > it away and use a very different, batch-style reordering code, of the > kind you can find in the freebidi library. The sole reason for > bidi.c's existence is to produce character positions in the _visual_ > order, one position at a time, something that no other bidi-aware > editor does. Yes, except we're not talking about "large chunks of text" here: somehow you went from "we need only a bunch of regexps to catch ligatures" to "we need to send entire paragraphs to the shaping engine, nothing less will do". My opinion is that we need a reasonable amount of context, often just a single character, and I see no reason to throw out the entire display engine because we want some look-ahead. > Moreover, not even the basic iteration, one level above bidi.c, where > we call get_next_display_element, then PRODUCE_GLYPHS, then > set_iterator_to_next -- not even that makes sense. Again, a single character of lookahead in the typical case, four characters for most ligatures; that doesn't push us over the line to "only a complete rewrite makes sense". > This basic loop > exists only because we examine characters one by one, switching from > buffer text to overlay or display strings, then back, as needed, and > applying faces as we go. Doing this in large chunks calls for a very > different structure of the code, and very different separation into > layers. Indeed. Which is why I'm not talking about doing it in large chunks, at this point. Let's keep doing it character by character but add what little we need to in order to look ahead a little. > This needs to be carefully designed in advance in a clean and > well-defined way, not lumped one patch upon another until it kinda > works... I agree "just start hacking on it with no understanding of the code until things appear to start working" is a bad strategy. So is "first, redesign the universe". To me, it seems like what I want is a reasonable compromise: not large chunks of text, because we can't do that, but some context, enough to do kerning and deal with ligatures. Remember that this discussion started when I mentioned that I was unhappy with HarfBuzz, and I still am, precisely because of its "first, send me your entire document" approach. I don't think it's the right approach to take this design flaw of HarfBuzz for granted and conclude that we need to rewrite the Emacs display engine before we can get English ligatures to display properly. If, that is, we can get look-ahead to work.