From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Akira Kyle Newsgroups: gmane.emacs.devel Subject: Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets) Date: Sat, 21 Nov 2020 20:35:52 -0700 Message-ID: <86pn46awrr.fsf@akirakyle.com> References: <864kmzupp0.fsf@akirakyle.com> Mime-Version: 1.0 Content-Type: text/plain; format=flowed Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="21461"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.4.13; emacs 28.0.50 To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Nov 22 12:06:17 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kgnCD-0005T9-0x for ged-emacs-devel@m.gmane-mx.org; Sun, 22 Nov 2020 12:06:17 +0100 Original-Received: from localhost ([::1]:57298 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kgnCC-0004Ua-1Q for ged-emacs-devel@m.gmane-mx.org; Sun, 22 Nov 2020 06:06:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:48748) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kggAb-0008I0-0R for emacs-devel@gnu.org; Sat, 21 Nov 2020 22:36:09 -0500 Original-Received: from mail-il1-f170.google.com ([209.85.166.170]:34864) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kggAS-000627-Bk for emacs-devel@gnu.org; Sat, 21 Nov 2020 22:36:08 -0500 Original-Received: by mail-il1-f170.google.com with SMTP id t13so12316413ilp.2 for ; Sat, 21 Nov 2020 19:35:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:subject :in-reply-to:date:message-id:mime-version; bh=nOxmF6Q7nLxJt3CvshmUV+UflUsyAbQL2M5CkrPMiY8=; b=b8EHAXRMDeOl1mkNzjRoM1aq9w1244nv3LoBANveP/dBFOv6NV8Yqk7+gIxIj6jxVY Dv3mBHpclVbaI3Th407NTvXqypoUimxyH3h9w4S2lJn/FATGglH7zyrffqJ1cv/CfhOi Hs9mL3d13x3lvW75VKhMSoRnv5Ig90mPqK4dXR1JwLEKpVioriFc5kRKFshCrih4hKvB x1l3c/64oUCYQc19774tWTf/ftlunXyiQDHKLIMfQ3Yq2ctkE+tg35x0e74gc3T++SpK j4GVXb1SZjFsuevCoLPdUDvCFp0m5UlVNL93eFZj4SW+RFCUjMsFL5e4/WHUI22x6D+S cWgQ== X-Gm-Message-State: AOAM5339Rf/1K0DI4rBgafpW8AcmY5FfqQsR4CVwW5fA6yrbM4cRA7Sj 2bxJhFkkdpW1qmqV9rUdZBoD7ZxvmYb0Tw== X-Google-Smtp-Source: ABdhPJx920e+kzfZNfAQ+nSp2mOJeZ+zQW5AzbI2ElHy2UKNzftit66g5XKU9wPhPeQfK713GdhBBw== X-Received: by 2002:a92:d311:: with SMTP id x17mr11037389ila.127.1606016153796; Sat, 21 Nov 2020 19:35:53 -0800 (PST) Original-Received: from data ([2601:281:8080:45f0:c135:9b95:ff:e6e2]) by smtp.gmail.com with ESMTPSA id v3sm914346ilj.28.2020.11.21.19.35.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 21 Nov 2020 19:35:53 -0800 (PST) In-reply-to: <864kmzupp0.fsf@akirakyle.com> Received-SPF: pass client-ip=209.85.166.170; envelope-from=aikokyle@gmail.com; helo=mail-il1-f170.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Sun, 22 Nov 2020 06:04:57 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:259603 Archived-At: Awhile ago I started a thread on rethinking the design of xwidgets, there was a lot of good discussion about the future of Emacs' rendering capabilities. It personally motivated me to try to better understand how Emacs' redisplay works and just overall better understand the C core of Emacs. As part of that exercise, I decided to see if I could port the existing xwidgets code into a dynamic module. The only xwidget Emacs currently implements is a webkit widget, so this effort turned into creating a dynamic module for embedding webkit in Emacs. It also happens that I've been wishing that I could never leave Emacs and a general web browser was the last daily app that I begrudgingly used outside of Emacs. Anyways here's the result: https://github.com/akirakyle/emacs-webkit.git (sorry for the github hosting, I just have yet to make the jump to another platform -- I guess what they say about vendor lock-in is true). I was able to implement all of the current features of the webkit xwidget plus some more. I think having this as a dynamic module makes it feel less hacked into Emacs (although still a bit hackey in some places). One interesting additional feature I was able to implement was support for TUI emacs. Essentially it opens a dedicated window for displaying webkit which can be controlled through the keybindings in the emacs buffer corresponding to that webkit view (of course one needs to have running a running graphical session pointed to in $DISPLAY). Perhaps when this code stabilizes, it might make sense to remove the experimental xwidget support from Emacs if this is deemed a worthy successor to such features. I also wanted to make a few more general comments prompted by the previous thread about the future of Emacs redisplay and rendering given what I've learned in this exercise. I now think its fairly foolish to try to integrate too much of a toolkit library's widgets into Emacs. There will always be this friction between Emacs and the toolkit library wanting to handle the business of drawing to the glass. In a lot of ways I've come away from this very impressed with how Emacs manages to have native toolkit ports to gtk, ns, win, and TUI rendering given the differences in how each wants to handle drawing to the screen, but on the other hand there's a lot of complexity and code behind making this happen, which presents a large barrier to entry to people like me who may want to hack on the lower level drawing capabilities of Emacs. We've also seen hardware change a lot since Emacs display machinery was written. CPUs have gotten faster and now everyone has a GPU. I think there's a potential for Emacs to become both snappier and more extensible with respect to its rendering. I had initially argued that Emacs probably shouldn't deal with rendering on the level of opengl, however I think as I've understood better the way Emacs currently handles rendering everything, that's exactly the direction Emacs should move in, *if it can*. So here is my new pie-in-the-sky proposal for what the future of Emacs rendering could look like. Given the success and hopefully soon-to-be in master native-comp branch, perhaps it may not be so crazy to think about moving more rendering to elisp. If a clean elisp interface to opengl were defined and Emacs were able to render itself using this optimized elisp opengl interface, the low level code for each platform could be minimized to just providing the opengl surface and exposing input events. In fact there may be ways in which elisp is well suited to the way modern GPU graphics are rendered using scene graphs. In fact gtk4 is moving its rendering to such an approach in order to take advantage of hardware graphics acceleration. I think a first step would likely be to focus on implementing an elisp interface to cairo as Emacs is already using more of cairo to render itself. If it seems like the performance is good and elisp can represent graphical operations well, than that might be a good indication that such a model could be successful path forward towards an Emacs that could bring its extensibility to its own basic rendering.