Removing the `window-id' frame parameter on non-Core Input platforms

Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

The only platform where it is useful is X with Core Input, where third
party programs can use it to manipulate a frame's X Window.

This is safe on Core Input, but on XInput 2 I'd like the freedom to
change the underlying X window of a frame after it is created to work
around some difficult GTK related problems, which will make continuing
to support the `window-id' parameter unreasonable.  Touch event tracking
on XInput2 also requires that no non-Emacs code modify a window created
by Emacs to work correctly.

On non-X platforms, `window-id' is just a pointer which is even more
useless, so I'd like to make this code in frame.c conditional on
HAVE_X_WINDOWS && !HAVE_XINPUT2:

  w = (uintptr_t) FRAME_NATIVE_WINDOW (f);
  store_in_alist (alistptr, Qwindow_id,
		  make_formatted_string (buf, "%"PRIuMAX, w));

Does that look OK to you?

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Po Lu <luangruo@yahoo.com> writes:

> The only platform where it is useful is X with Core Input, where third
> party programs can use it to manipulate a frame's X Window.
>
> This is safe on Core Input, but on XInput 2 I'd like the freedom to
> change the underlying X window of a frame after it is created to work
> around some difficult GTK related problems, which will make continuing
> to support the `window-id' parameter unreasonable.  Touch event tracking
> on XInput2 also requires that no non-Emacs code modify a window created
> by Emacs to work correctly.
>
> On non-X platforms, `window-id' is just a pointer which is even more
> useless, so I'd like to make this code in frame.c conditional on
> HAVE_X_WINDOWS && !HAVE_XINPUT2:
>
>   w = (uintptr_t) FRAME_NATIVE_WINDOW (f);
>   store_in_alist (alistptr, Qwindow_id,
> 		  make_formatted_string (buf, "%"PRIuMAX, w));
>
> Does that look OK to you?

The problem this proposal was supposed to solve turned out to be
specific to a version of GNOME Shell, so I added an entry in
etc/PROBLEMS instead.

Please disregard this message.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Date: Sat, 18 Dec 2021 13:29:48 +0800
> 
> The only platform where it is useful is X with Core Input, where third
> party programs can use it to manipulate a frame's X Window.
> 
> This is safe on Core Input, but on XInput 2 I'd like the freedom to
> change the underlying X window of a frame after it is created to work
> around some difficult GTK related problems, which will make continuing
> to support the `window-id' parameter unreasonable.  Touch event tracking
> on XInput2 also requires that no non-Emacs code modify a window created
> by Emacs to work correctly.

Please describe the problems you see in continuing the support for
window-id.  You say it's unreasonable, but please tell more to clarify
why it is so.

I don't necessarily object to the idea, but there could be in
principle other alternatives that are less backward-incompatible.  I'd
like us to weigh those before we make the decision.

Thanks.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Note that the problem this is supposed to fix is specific to a single
version of a single window manager, so I installed an entry in
etc/PROBLEMS instead, but this could still be a valid concern if the
problem resurfaces in the future

Eli Zaretskii <eliz@gnu.org> writes:

> Please describe the problems you see in continuing the support for
> window-id.  You say it's unreasonable, but please tell more to clarify
> why it is so.

Essentially, to work around the freeze described in PROBLEMS, a frame's
window is recreated which changes the XID of the frame.  Non-Emacs X
clients that have kept track of the previous XID will crash with a
BadValue error when trying to use the previous XID.

Other uses of the `window-id' parameter such as to display windows as
children of a frame are discouraged on XInput2 builds, as it can
interfere with XInput2 event handling (but this is a theoretical
problem, I have yet to see it happen in practice to Emacs) in
complicated ways, such as unexpected touch sequence ownership changes
leading to Emacs crashes.

Thanks.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Sat, 18 Dec 2021 15:53:29 +0800
> 
> Essentially, to work around the freeze described in PROBLEMS, a frame's
> window is recreated which changes the XID of the frame.  Non-Emacs X
> clients that have kept track of the previous XID will crash with a
> BadValue error when trying to use the previous XID.

But that isn't Emacs-specific, is it?  AFAIR, the window ID is a
general X feature, we just support it.

> Other uses of the `window-id' parameter such as to display windows as
> children of a frame are discouraged on XInput2 builds, as it can
> interfere with XInput2 event handling

You mean, using it for our child frames?

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> But that isn't Emacs-specific, is it?  AFAIR, the window ID is a
> general X feature, we just support it.

Yes, but after working around the fullscreen freeze bug, we can no
longer longer guarantee that a frame's XID will remain constant (and
therefore useful).

>> Other uses of the `window-id' parameter such as to display windows as
>> children of a frame are discouraged on XInput2 builds, as it can
>> interfere with XInput2 event handling

> You mean, using it for our child frames?

No, other processes using it to place their windows onto our frames.

The problem is that XInput2 touch support was designed to assume that a
single client owns a window and all its children.  Introducing another
process and therefore client into the window hierarchy causes all kinds
of race conditions when two processes try to take ownership of the same
touch sequence at once, and so on and so forth.

Thanks.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Stefan Monnier]

> The problem is that XInput2 touch support was designed to assume that a
> single client owns a window and all its children.  Introducing another
> process and therefore client into the window hierarchy causes all kinds
> of race conditions when two processes try to take ownership of the same
> touch sequence at once, and so on and so forth.

Maybe we should deprecate it, then.  I have used this frame parameter in
scripts that pass it to external applications in order to record
a video, tho, so I'm not sure what we should replace it with.


        Stefan

RE: [External] : Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Drew Adams]

> Maybe we should deprecate it, then.  I have used this frame parameter in
> scripts that pass it to external applications in order to record
> a video, tho, so I'm not sure what we should replace it with.

FWIW, I use frame parameter `window-id'.

I use it, for example, to sort stacking
of thumbnail frames by window ID.

In practice, it seems to be an OK proxy
for time of window creation, for example.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I have used this frame parameter in scripts that pass it to external
> applications in order to record a video, tho, so I'm not sure what we
> should replace it with.

Does that use case involve creating a separate display connection, and
selecting events from the Emacs window?  If so, it will not work
correctly on XInput 2.2 or later.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Stefan Monnier]

Po Lu [2021-12-19 10:48:02] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I have used this frame parameter in scripts that pass it to external
>> applications in order to record a video, tho, so I'm not sure what we
>> should replace it with.
>
> Does that use case involve creating a separate display connection, and
> selecting events from the Emacs window?  If so, it will not work
> correctly on XInput 2.2 or later.

I have no idea.  I just passed it as argument to `recordmydesktop` and
it did its thing somehow.


        Stefan

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Po Lu [2021-12-19 10:48:02] wrote:
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> I have used this frame parameter in scripts that pass it to external
>>> applications in order to record a video, tho, so I'm not sure what we
>>> should replace it with.
>>
>> Does that use case involve creating a separate display connection, and
>> selecting events from the Emacs window?  If so, it will not work
>> correctly on XInput 2.2 or later.
>
> I have no idea.  I just passed it as argument to `recordmydesktop` and
> it did its thing somehow.

Please test that program with an XInput 2 build: it is vital to know how
third-party programs are interacting with Emacs windows in-the-wild.

Thanks.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Akira Kyle]

On Fri, Dec 17, 2021 at 10:31 PM Po Lu <luangruo@yahoo.com> wrote:
>
> The only platform where it is useful is X with Core Input, where third
> party programs can use it to manipulate a frame's X Window.
>
> This is safe on Core Input, but on XInput 2 I'd like the freedom to
> change the underlying X window of a frame after it is created to work
> around some difficult GTK related problems, which will make continuing
> to support the `window-id' parameter unreasonable.  Touch event tracking
> on XInput2 also requires that no non-Emacs code modify a window created
> by Emacs to work correctly.
>
> On non-X platforms, `window-id' is just a pointer which is even more
> useless, so I'd like to make this code in frame.c conditional on
> HAVE_X_WINDOWS && !HAVE_XINPUT2:
>
>   w = (uintptr_t) FRAME_NATIVE_WINDOW (f);
>   store_in_alist (alistptr, Qwindow_id,
>                   make_formatted_string (buf, "%"PRIuMAX, w));
>
> Does that look OK to you?
>

emacs-webkit uses the pointer obtained via 'window-id' on pgtk builds
to help identify the correct gtk container widget containing contents
of the emacs frame that should become the parent of the webkit widget.
It's a bit of a hack but would be potentially more difficult without
this pointer.

I also know that EAF uses the 'widow-id' parameter on x+gtk builds in
order to allow Qt widgets to be reparented onto emacs frames.

Also a slight tangent, but Po, do you plan to make xwidgets work on
pgtk? I'm wondering that since you've been trying to revive the
xwidget code and fixed the flickering issues on x+gtk, that xwidgets
are looking more like they are here to stay as a potentially
non-experimental feature? If that's the case, then I think I'd like to
then just integrate the work I've done on emacs-webkit into a pgtk
version of xwidgets? I think one of the key differences between the
codes are the osr versus one-to-one buffer-webkit view models.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Akira Kyle <akira@akirakyle.com> writes:

> emacs-webkit uses the pointer obtained via 'window-id' on pgtk builds
> to help identify the correct gtk container widget containing contents
> of the emacs frame that should become the parent of the webkit widget.
> It's a bit of a hack but would be potentially more difficult without
> this pointer.

I'm not sure we want to allow direct access to the innards of how Emacs
utilises GTK widgets.  That way lies madness.

> I also know that EAF uses the 'widow-id' parameter on x+gtk builds in
> order to allow Qt widgets to be reparented onto emacs frames.

I've already discussed this with the EAF developers.  Until Qt gains the
ability to not select for TouchBegin, TouchUpdate and TouchEnd, EAF will
likely not work on XInput2 builds.

> Also a slight tangent, but Po, do you plan to make xwidgets work on
> pgtk? I'm wondering that since you've been trying to revive the
> xwidget code and fixed the flickering issues on x+gtk, that xwidgets
> are looking more like they are here to stay as a potentially
> non-experimental feature? If that's the case, then I think I'd like to
> then just integrate the work I've done on emacs-webkit into a pgtk
> version of xwidgets? I think one of the key differences between the
> codes are the osr versus one-to-one buffer-webkit view models.

I will work on it eventually, which will be before the release of Emacs
29.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Po Lu <luangruo@yahoo.com> writes:

>> then just integrate the work I've done on emacs-webkit into a pgtk
>> version of xwidgets? I think one of the key differences between the
>> codes are the osr versus one-to-one buffer-webkit view models.

> I will work on it eventually, which will be before the release of Emacs
> 29.

Also, BTW, I don't think you will have trouble integrating the
emacs-webkit work into the X version of xwidgets.

Most of the offscreen rendering and event passthrough between X windows
and the WebKit widget is done transparently, so you essentially don't have
to worry about it, as things should "just work".

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Akira Kyle]

On Sun, Dec 19, 2021 at 8:20 PM Po Lu <luangruo@yahoo.com> wrote:
>
> >> then just integrate the work I've done on emacs-webkit into a pgtk
> >> version of xwidgets? I think one of the key differences between the
> >> codes are the osr versus one-to-one buffer-webkit view models.
>
> > I will work on it eventually, which will be before the release of Emacs
> > 29.
>
> Also, BTW, I don't think you will have trouble integrating the
> emacs-webkit work into the X version of xwidgets.
>
> Most of the offscreen rendering and event passthrough between X windows
> and the WebKit widget is done transparently, so you essentially don't have
> to worry about it, as things should "just work".

Except the user experience is different. Frankly I find it not very
helpful to have the same exact view into a buffer on two windows which
is what the osr technique ends up accomplishing. The fact that
scrolling in one xwidget webkit buffer scrolls in every other I find
to be distracting. Also the case of unequal sized windows displaying
the same webkit buffer is still not handled correctly as the larger
buffer will have it's view clipped to the size of the smaller in focus
buffer.

Another feature not currently "just working" transparently is the
webkit inspector. It currently seg faults emacs when the webkit buffer
is killed and is impossible to interact with before then. Perhaps the
osr technique can be made to work in these cases, but I view it as
really going against the grain of the gtk widget (and cocoa) model.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Akira Kyle <akira@akirakyle.com> writes:

> Except the user experience is different.

You don't have to display the same widget in two windows, if that's what
you're concerned about.

> Frankly I find it not very helpful to have the same exact view into a
> buffer on two windows which is what the osr technique ends up
> accomplishing. The fact that scrolling in one xwidget webkit buffer
> scrolls in every other I find to be distracting. Also the case of
> unequal sized windows displaying the same webkit buffer is still not
> handled correctly as the larger buffer will have it's view clipped to
> the size of the smaller in focus buffer.

It correctly fits the model of Emacs display, which is the main concern.

> Another feature not currently "just working" transparently is the
> webkit inspector. It currently seg faults emacs when the webkit buffer
> is killed and is impossible to interact with before then.

On X11?  It works for me, and if it doesn't, that is a bug.  Please send
a backtrace along with the WebKitGTK version you are using.

Thanks.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Akira Kyle]

On Sun, Dec 19, 2021 at 8:39 PM Po Lu <luangruo@yahoo.com> wrote:
>
> > Except the user experience is different.
>
> You don't have to display the same widget in two windows, if that's what
> you're concerned about.
>
> > Frankly I find it not very helpful to have the same exact view into a
> > buffer on two windows which is what the osr technique ends up
> > accomplishing. The fact that scrolling in one xwidget webkit buffer
> > scrolls in every other I find to be distracting. Also the case of
> > unequal sized windows displaying the same webkit buffer is still not
> > handled correctly as the larger buffer will have it's view clipped to
> > the size of the smaller in focus buffer.
>
> It correctly fits the model of Emacs display, which is the main concern.

I would disagree. To fit the Emacs display model I think xwidget
webkit buffers should behave the same as eww buffers, where one can
independently scroll two windows containing the same buffer (ie one
could view the content at the top of a page in one buffer and the
content at the bottom in another which can be very useful).

> > Another feature not currently "just working" transparently is the
> > webkit inspector. It currently seg faults emacs when the webkit buffer
> > is killed and is impossible to interact with before then.
>
> On X11?  It works for me, and if it doesn't, that is a bug.  Please send
> a backtrace along with the WebKitGTK version you are using.

This was via xwayland. I'll try to come up with a recipe to reproduce
and send a bug report.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Akira Kyle <akira@akirakyle.com> writes:

> I would disagree. To fit the Emacs display model I think xwidget
> webkit buffers should behave the same as eww buffers, where one can
> independently scroll two windows containing the same buffer (ie one
> could view the content at the top of a page in one buffer and the
> content at the bottom in another which can be very useful).

Why is that so?  xwidgets aren't text, they're special display elements
like images.

> This was via xwayland. I'll try to come up with a recipe to reproduce
> and send a bug report.

It should work the same regardless of whether Xwayland is involved.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Akira Kyle]

On Sun, Dec 19, 2021 at 9:45 PM Po Lu <luangruo@yahoo.com> wrote:
>
> Akira Kyle <akira@akirakyle.com> writes:
>
> > I would disagree. To fit the Emacs display model I think xwidget
> > webkit buffers should behave the same as eww buffers, where one can
> > independently scroll two windows containing the same buffer (ie one
> > could view the content at the top of a page in one buffer and the
> > content at the bottom in another which can be very useful).
>
> Why is that so?  xwidgets aren't text, they're special display elements
> like images.

But even with a buffer showing an image, I can scroll it independently
in two windows, so xwidgets don't even behave like images. I would
expect displaying an HTML document that only contains unformatted text
to behave similarly in both eww and xwidgets webkit with respect to
scrolling the document. The current behavior of xwidget webkit with
respect to scrolling doesn't appear to me to have any analog to
anything else in emacs, perhaps the closet would be follow-mode if the
two windows displayed the same buffer location.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Akira Kyle <akira@akirakyle.com> writes:

> But even with a buffer showing an image, I can scroll it independently
> in two windows, so xwidgets don't even behave like images.

If you play a GIF that depicts a scrolling WebKitWebView, you can't
really "scroll" it independently either.  Ditto for xwidgets.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Akira Kyle]

On Mon, Dec 20, 2021 at 6:15 PM Po Lu <luangruo@yahoo.com> wrote:
>
> > But even with a buffer showing an image, I can scroll it independently
> > in two windows, so xwidgets don't even behave like images.
>
> If you play a GIF that depicts a scrolling WebKitWebView, you can't
> really "scroll" it independently either.  Ditto for xwidgets.

I would argue that most users would expect scrolling an xwidget webkit
buffer that is visible in two windows to behave like eww, not like
advancing a frame in a GIF. In fact I would further argue that the way
emacs handles displaying a GIF also does not fit the paradigm for
emacs buffers and windows. That paradigm being that each window is a
separate view into a buffer where the position in the buffer that the
window displays is independent of any other window (ie is window
local). If, for a GIF, I reasonably say that a buffer's position in a
GIF should be its frame number, then I would expect the above
buffer-window paradigm to hold. Yet currently it does not. This in
contrast to what I would say would be a conceptually similar situation
in doc-view which does obey this paradigm.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Po Lu]

Akira Kyle <akira@akirakyle.com> writes:

>> If you play a GIF that depicts a scrolling WebKitWebView, you can't
>> really "scroll" it independently either.  Ditto for xwidgets.

> I would argue that most users would expect scrolling an xwidget webkit
> buffer that is visible in two windows to behave like eww, not like
> advancing a frame in a GIF.  That paradigm being that each window is a
> separate view into a buffer where the position in the buffer that the
> window displays is independent of any other window (ie is window
> local).

The only paradigm I see is that a single glyph (such as an xwidget
glyph, produced by a display property) should be able to display in
multiple windows, which this implementation (and the GIF display as
well) fulfils.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Akira Kyle]

On Mon, Dec 20, 2021 at 9:38 PM Po Lu <luangruo@yahoo.com> wrote:
>
> > I would argue that most users would expect scrolling an xwidget webkit
> > buffer that is visible in two windows to behave like eww, not like
> > advancing a frame in a GIF.  That paradigm being that each window is a
> > separate view into a buffer where the position in the buffer that the
> > window displays is independent of any other window (ie is window
> > local).
>
> The only paradigm I see is that a single glyph (such as an xwidget
> glyph, produced by a display property) should be able to display in
> multiple windows, which this implementation (and the GIF display as
> well) fulfils.

Yes that's the obvious paradigm with respect to the implementation
details of emacs' display. I'm trying to argue purely in terms of what
a user might expect given the way normal text buffers behave in emacs.
With an xwidget webkit window where the single glyph is sized to
always fill the window (however it often gets confused on which window
to size when there are multiple windows of different sizes displaying
it), as a user I don't know or care that the window is only displaying
a single glyph. I see what looks like a window displaying a buffer's
contents in a pretty way (just like eww or doc-view), then I would
expect the mechanics of scrolling to be similar, however I am
surprised when they don't behave in the way I've come to expect for
other emacs modes that display some "rendered" version of the raw
buffer contents.

Re: Removing the `window-id' frame parameter on non-Core Input platforms

[Akira Kyle]

On Sun, Dec 19, 2021 at 8:16 PM Po Lu <luangruo@yahoo.com> wrote:
>
> I'm not sure we want to allow direct access to the innards of how Emacs
> utilises GTK widgets.  That way lies madness.

Well I don't think you can stop it so long as dynamic module code runs
in the same process as that of the emacs GUI. But fair, I don't claim
what emacs-webkit does is anything more than a "clever" hack as a
proof of concept.