From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Akira Kyle Newsgroups: gmane.emacs.devel Subject: Re: Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets) Date: Sat, 28 Nov 2020 18:29:08 -0700 Message-ID: <86o8jh9cij.fsf@akirakyle.com> References: <864kmzupp0.fsf@akirakyle.com> <86pn46awrr.fsf@akirakyle.com> <87y2ise7j5.fsf@gnus.org> <86tutfwhr6.fsf@akirakyle.com> <87h7pfb76z.fsf@gnus.org> <86h7peqkt5.fsf@akirakyle.com> <83tutdsc8i.fsf@gnu.org> <86eeker01y.fsf@akirakyle.com> <83wny5naf5.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; format=flowed Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="26822"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.4.13; emacs 28.0.50 Cc: larsi@gnus.org, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Nov 29 02:30:14 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kjBXa-0006tf-QU for ged-emacs-devel@m.gmane-mx.org; Sun, 29 Nov 2020 02:30:14 +0100 Original-Received: from localhost ([::1]:36836 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kjBXZ-0008TE-Om for ged-emacs-devel@m.gmane-mx.org; Sat, 28 Nov 2020 20:30:13 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41078) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kjBWd-00081N-7B for emacs-devel@gnu.org; Sat, 28 Nov 2020 20:29:15 -0500 Original-Received: from mail-io1-f47.google.com ([209.85.166.47]:35464) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kjBWb-0006te-5e; Sat, 28 Nov 2020 20:29:14 -0500 Original-Received: by mail-io1-f47.google.com with SMTP id i9so8324197ioo.2; Sat, 28 Nov 2020 17:29:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version; bh=YpNjjZPc5qW+mhXgV8n3y2xhQXYIFiQZc2jK0juCupc=; b=nV13dRDuATVYtqsMzpDECHv414i6suLdnYtMn+uiIcTTORPai1tBG1YQMad7Ow4/v5 9MckO+47ymj7gAoBeUHJNsUfKcZxmG7BC8Umm7SbPPUf1IInRUDPRIe1IJq2q4Sx9XmZ pDRhiB2YhRTP9Fot06fwbz4BlOzcAcDZ6Img3bYaxirGwQHxV69F9oFZJSKWHwYYmWPs cXojl9uZXnxkKFA7Cz1skN6ml3SwysnsbXrrTLLCOwrf5gpBI84++GjHchkBL5BaaLUn mv3NI1s2oF981b7OarAC0CPPG8VBFfQiDAsK5SuMXCTPPlujxJPTreiwGYuDaj3Mua2N OY+A== X-Gm-Message-State: AOAM530yOkGrAYDX/McJHStQnyuixEYn0zphFHWUYXgDIYlNyyJFmmb9 gG/FQiBDnFPXCAug+0tNRjUYpHcy1n8IrofJ X-Google-Smtp-Source: ABdhPJwPaxPnD7/EgjhmohScH2cxH/TLRsEG0uEBB5ERx2fZYV7e1jMsOKTnVXQfxKqb3LQW/NNu9w== X-Received: by 2002:a6b:6f07:: with SMTP id k7mr550957ioc.48.1606613350576; Sat, 28 Nov 2020 17:29:10 -0800 (PST) Original-Received: from data ([2601:281:8080:45f0:b07f:c339:176b:ebf2]) by smtp.gmail.com with ESMTPSA id c89sm8152394ilf.26.2020.11.28.17.29.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Nov 2020 17:29:09 -0800 (PST) In-reply-to: <83wny5naf5.fsf@gnu.org> Received-SPF: pass client-ip=209.85.166.47; envelope-from=aikokyle@gmail.com; helo=mail-io1-f47.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:259984 Archived-At: On Sat, Nov 28, 2020 at 01:38 AM, Eli Zaretskii wrote: >> > It is a relatively simple matter to disable some >> > optimizations >> > when a widget is displayed. It is also relatively simple to >> > tell >> > the widget to redraw itself when redisplay changed its >> > position on >> > the screen. >> >> Yes, but you had previously pointed to that fact that xwidgets >> needs to such disable such optimizations as part of the >> "kludge" >> that would need to be resolved for xwidgets to be considered >> viable for a path forward. Having looked at that, I'm not sure >> gtk >> will ever want to cooperate without doing things that would be >> considered hacks by gtk folks. > > I'd appreciate more details on the GTK part of this. I only > said that > from the Emacs display engine side things are not too hairy. > from my > POV, xwidgets integration simply stopped short of going all the > way > towards integration into the redisplay framework, and I don't > think > anyone said before that this was done because better integration > was > impossible or very hard. One such fundamental incompatibility stems from the fact that Emacs may have multiple views on one buffer split across various windows and/or frames. Gtk assumes that each widget will only have one view. The 'webkit xwidget type currently works around this by drawing to an offscreen window then for every window that's supposed to display the widget, creating a gtk drawing area widget and copying the offscreen window's surface to the drawing surface. I don't think gtk really intends for ofscreen widgets to be used this way and as far as I can tell offscreen window widgets have been removed in gtk4. Another issue is that emacs uses a GtkFixed (or a custom subclassed EmacsFixed for gtk3) as the container widget for the main frame area. There isn't a mechanism to control z-ordering of child widgets other than removing and re-adding them in the desired z-order, which is what emacs-webkit has to do in order for child frames to not render below the webkit widget. >> > I don't see why you consider this such a significant problem. >> > The >> > number of interfaces that such a "rich" display element needs >> > to >> > support is very small and well-defined. The problem with >> > xwidgets >> > in this regard is that its interaction with the display code >> > was >> > left unfinished, it basically just copy/pastes the code which >> > supports display of images, which is not 100% correct, since >> > xwidgets don't display a static bitmap. >> >> The "number interfaces" will likely be small no matter what >> solution is taken, its more about how easy you want to make it >> for >> a "rich" display element to really mess with Emacs' display and >> input handling. For example with xwidgets the onus is on the >> widget implementation to not totally steal keyboard focus from >> Emacs. I've found that gtk widgets really like stealing >> keyboard >> focus so its easy to end up in a state you can't escape out >> of. > > Keyboard focus is a separate issue. I only talked about > integration > with redisplay. > > For keyboard input integration, we'd need to develop some > infrastructure, if the existing one doesn't provide enough > hooks. If > you or someone else can describe the details, we could discuss > that in > more practical terms. I think it's too soon to give up on the > practical possibility to solve these problems before we > discussed > that, or are even fully aware of the problems and the potential > solutions. > >> Similarly gtk widgets won't always easily draw themselves >> where you want at the size you want so its easy for widgets to >> render, say, across window boundaries. > > If that is true, then your method of letting the widget display > in a > window won't work well either, right? IOW, whether the screen > area > given to the widget is a full Emacs window or just its part is > not an > important aspect of the integration of such widgets into the > Emacs > display. > >> The xwidget approach requires non-insignificant work on the >> widget >> end to ensure the widgets will work within the confines of >> Emacs' >> redisplay. > > As long as the widget can be told to use a given rectangular > portion > of the screen, we should be fine: the current Emacs display > engine is > perfectly capable of handling display elements that occupy an > area of > certain dimensions. This is more an issue on the gtk side of things. With the webkit widget it isn't an issue since the widget doesn't define a minimum size so any size you request it to render at, it will obey. However other widgets, such as buttons, sliders, etc, have a minimum size they will render at. If you tell it to render itself smaller, it will refuse and so it's impossible to simply tell the widget "here's the space you have to go in". This becomes a problem when the widget needs to be clipped at a window boundary as gtk will happily draw the widgets across them as gtk has no internal knowledge of Emacs window boundaries being a place that widget must be clipped at. Thus every widget that will live in a buffer will need to be inside a custom container than handles this clipping for the widget. >> I'd think that such an interface to support "rich" display >> elements >> should be restrictive enough to not easily allow the "rich" >> display >> elements to misbehave. > > Please tell more about the reasons for this conclusion. I don't > yet > see the difficulties you envision. While all these issues can be overcome in one way or another, I think they point at the differences both the UI paradigms of Emacs and gtk and the subsequent differences in how each want to draw to the screen. In addition to the above specific examples of difficulties I've encountered so far with making gtk widgets work well with Emacs' display, there's the fundamental problem of this route for "richer display elements" not being portable. It'll always be confined to Emacs builds `--with-x` (or `--with-pgtk` now). While I feel this is fine for emacs-webkit, I think it would be advantageous to not tie such a feature to gtk itself. The direction I'm thinking of going in the future since I feel like thus far the xwidget route has been fairly troublesome is to expose interfaces to the cairo surfaces used in image.c. This could be through modifying image.c or through defining a new display element type. I would try to expose the cairo API to lisp and allow dynamic modules to directly draw to a cairo surface given to it through the lisp interface. Gtk along with many other libraries (such as librsvg and poppler) support drawing to a cairo surface and then Emacs gets to have full control over how that cario surface ends up on screen. I would see this as a stepping stone to attempting to do something similar with a opengl surface.