From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Akira Kyle Newsgroups: gmane.emacs.devel Subject: Re: Rethinking the design of xwidgets Date: Tue, 13 Oct 2020 11:05:51 -0600 Message-ID: <86v9fet5sg.fsf@akirakyle.com> References: <864kmzupp0.fsf@akirakyle.com> <835z7e2ouj.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; format=flowed Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="11255"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.4.13; emacs 28.0.50 Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Oct 13 19:07:50 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kSNmA-0002ob-8W for ged-emacs-devel@m.gmane-mx.org; Tue, 13 Oct 2020 19:07:50 +0200 Original-Received: from localhost ([::1]:55464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kSNm9-0006gi-8D for ged-emacs-devel@m.gmane-mx.org; Tue, 13 Oct 2020 13:07:49 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:33106) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kSNkN-0005HA-7d for emacs-devel@gnu.org; Tue, 13 Oct 2020 13:05:59 -0400 Original-Received: from mail-il1-f171.google.com ([209.85.166.171]:46678) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kSNkK-0007lV-Uq; Tue, 13 Oct 2020 13:05:58 -0400 Original-Received: by mail-il1-f171.google.com with SMTP id l16so725026ilt.13; Tue, 13 Oct 2020 10:05:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version; bh=XNfZR+LcAUd/6zxdduuLr7+Vj0pY50KrWIdmyD6SoMo=; b=ojPuwWBjxnUi6rSexheTxZJwv/09wb/2f3nE5IVQt7mPodVQUDKNtjxCzRtbHLPuru yap5LVhnrkdeH5ijCXX0zJB5/4+QGe5rVQmir62Lvd76WSoSn/hPgBAQY0tS3eFLg1mN Z1dNLii9TK1Bq9M0HrR0SpIqV9pCY5pE30PsZ2jeK/eqs699lVqqWAt6Qm7TPADRvTnX nsd9wC+R5OELKfD67NOoL/XfrJU4LPKZ62DuOO/xx3Vf9OMLV1GILh/LLfNoYw00Feup Tt2Y61LQAb5uKYDoL6Mvf7zfru0EAiBue6VXcrhOLM87o1rrfsXmVX4RSez2GJmm3WrC HDsg== X-Gm-Message-State: AOAM531jYUg9VTu/7TAUziqRttxTyYmpnIjqEt6iryYzrS67KODvsKek BYUxtgdaVWijE+gmvAlFV1axLeiOLV4AVw== X-Google-Smtp-Source: ABdhPJxLGRiDH1prulcNnkcSsNIY5pQZqTTEghnN8hXhxAjcwGoD5lkE0Ao0/yYK9cwhz3T3zDaSPg== X-Received: by 2002:a92:de07:: with SMTP id x7mr695938ilm.33.1602608753000; Tue, 13 Oct 2020 10:05:53 -0700 (PDT) Original-Received: from lore ([2601:281:8080:45f0:ca96:c97a:d919:de97]) by smtp.gmail.com with ESMTPSA id y12sm386015ilk.56.2020.10.13.10.05.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2020 10:05:52 -0700 (PDT) In-reply-to: <835z7e2ouj.fsf@gnu.org> Received-SPF: pass client-ip=209.85.166.171; envelope-from=aikokyle@gmail.com; helo=mail-il1-f171.google.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/13 13:05:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:257545 Archived-At: On Tue, Oct 13, 2020 at 08:16 AM, Eli Zaretskii wrote: > Can you tell more about the dynamic content you had in mind, and > why > you thought to use xwidgets as the basis for that? There's IMO > a leap > of logic here that I cannot follow, perhaps because I don't have > the > necessary background information. Sure, so to elaborate on my motivation: I spend a lot of my time in emacs using org-mode's babel support for python, specifically through emacs-jupyter [1]. This gives me much of the power of Jupyter notebooks, however not all. For example when generating plots using matplotlib, they are simply embedded in the buffer as an image, however matplotlib also has support for interactive backends [2] which allow one to zoom and pan the plot and see it update in real time as each plot command is issued. It would be convenient to have such interactivity directly inside the buffer rather than having to use an external window, similar to the interactivity offered when using matplotlib in a jupyter notebook. The bokeh plotting [3] library provides even more interactivity, allowing one to create complex and dynamic and interactive visualizations. When used inside a jupyter notebook, the plot is embedded inline, however when used with emacs-jupyter, it launches an external browser to display the interactive plot and it's controls. It would be fantastic if I could have such interactivity inside Emacs. Another use case I had in mind for my proposed improved xwidgets is better pdf rendering. I use pdf-tools constantly and it's one of the best pdf viewers out there (not just because it's in emacs). I'll sometimes even use it to give presentations so I never have to leave Emacs. However it's design inherently limits its rendering performance as it must rasterize each pdf page in a separate process then pipe that data to emacs to display as an image. Granted this could be avoided by turning the epdfinfo server process into a dynamic module, however it will still be a bit roundabout as the pdf page would have to first be rendered into a cairo context then rasterized then Emacs will draw that rasterized image onto another cairo context to get it onscreen. Having direct access to an onscreen cairo context would eliminate this and make it easier to support crisp, performant rendering onto HiDPI displays. Maybe I could even throw slide animations into my presentations this way! > From my POV, xwidgets have several problems which don't allow me > to > easily see them as the basis for some much more general set of > features or infrastructure: > > . they depend on GTK and webkit, whose development leaves a > lot to > be desired, and some say parts of that have no future; they > are > definitely less portable than I'd like to see in Emacs My proposal should actually make it easier for xwidgets to be decoupled from a specific GUI toolkit. The recent addition for a cocoa webkit xwidget has forced me to think a bit about this as I modify the xwidget code, and I'm fairly convinced that xwidgets can be designed is such as way that any object-based and event-driven toolkit could fit in its abstraction. > . their integration in the Emacs display code "needs work", > there's > at least one or two places where the code which handles them > is > clearly wrong -- this is semi-okay for a minor niche > feature, but > not for something on which we want to build our future Could you point me to some of those places. Maybe they are issues I've already taken a look at. > So before we decide that xwidgets is the way to go, I'd like to > make a > step back and see what other alternatives we can think of, if > any, and > carefully consider which alternative is best in the long run. I'd be interested if you can think of any alternatives since, at least to me, this seems like the natural way to implement such features. > Here, too, I'd like first to consider a design that doesn't > require > dynamic modules as a prerequisite for the feature. If we can > come up > with something that requires only Lisp (like image display does, > for > example), then I think we all win, because it is much easier to > write > a Lisp package than a package that requires a module in C or > similar > language. I think my pdf use case illustrates the issue with Emacs trying to support every such type of content as a high level "image-like" feature since you either force the application to convert everything into a sub-optimal format that Emacs understands for display or Emacs' must add native for such format, increasing the number of external dependencies to build Emacs. For example, consider svg support, which is my preferred image format. As far is I can tell, currently Emacs rasterizes svgs itself before ultimately drawing them on a cairo context. Since librsvg supports drawing directly on a cairo context, it would be much more efficient to do so, however adding support for svg could be easily moved out of emacs into a dynamic module and drawn directly onto it's own widget. This would allow users of the NS toolkit to not have to rely on librsvg if they didn't want to as I believe NS can render svgs itself. > Bottom line, I'd like to understand the problem you are trying > to > solve better before we conclude that xwidgets and dynamic > modules are > the way to go. Let me know if there's anything more I can clarify about my motivation and proposal to improve xwidgets. [1] https://github.com/nnicandro/emacs-jupyter [2] https://matplotlib.org/faq/usage_faq.html#what-is-a-backend [3] https://bokeh.org/