bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Platon Pronko]

This looks like Xmonad-specific regression after commit 483c5e95 by Martin Rudalics.

Steps to reproduce (under Xmonad):

1. Open a new frame (./src/emacs -Q);
2. Type something in a buffer;
3. Switch to a different workspace;
4. Switch back;
5. Observe buffer being blank and not responding to any interaction.

I bisected this problem to commit 483c5e95 (from 2021-05-05). The trouble seems to be caused by this change in xterm.c:

@@ -8232,33 +8238,36 @@ handle_one_xevent (struct x_display_info *dpyinfo,
            if (!FRAME_VISIBLE_P (f))
              {
                block_input ();
-              SET_FRAME_VISIBLE (f, 1);
-              SET_FRAME_ICONIFIED (f, false);
-              if (FRAME_X_DOUBLE_BUFFERED_P (f))
+             /* The following two are commented out to avoid that a
+                plain invisible frame gets reported as iconified.  That
+                problem occurred first for Emacs 26 and is described in
+                https://lists.gnu.org/archive/html/emacs-devel/2017-02/msg00133.html.  */
+/**          SET_FRAME_VISIBLE (f, 1); **/
+/**          SET_FRAME_ICONIFIED (f, false); **/
+
+             if (FRAME_X_DOUBLE_BUFFERED_P (f))
                  font_drop_xrender_surfaces (f);
                f->output_data.x->has_been_visible = true;
                SET_FRAME_GARBAGED (f);

Specifically the commenting of SET_FRAME_VISIBLE(f, 1) causes the problem (if I uncomment that line, buffer redisplay starts to work correctly).

Trying the same with XFCE doesn't give the error. I compared the sequences of X events received under different WMs, here's the lists (without "other", "MotionNotify" and key events, for brevity):

Xmonad:
1. When switching to different workspace:
    - FocusOut x3
    - UnmapNotify x1
    - LeaveNotify x2
    - PropertyNotify x2
2. When switching back:
    - MapNotify x1
    - VisibilityNotify x3
    - Expose x4
    - EnterNotify x2
    - FocusIn x2
    - ClientMessage x1
    - PropertyNotify x1
    - FocusIn x1

XFCE:
1. When switching to different workspace:
    - FocusOut x2
    - FocusIn x1
    - FocusOut x2
    - LeaveNotify x2
    - UnmapNotify x1
    - PropertyNotify x3
2. When switching back:
    - VisibilityNotify x3
    - Expose x4
    - EnterNotify x2
    - PropertyNotify x2
    - FocusIn x2
    - ClientMessage x1
    - PropertyNotify x2
    - FocusOut x2
    - FocusIn x1
    - FocusOut x2
    - FocusIn x2

Further inspection of SET_FRAME_VISIBLE calls shows that under Xmonad first the frame is set to invisible when UnmapNotify event happens, then after switching back the only time when it could have been set to "visible" was in second "Expose" event (but this line is commented out in commit 483c5e95, so frame thinks it's invisible and thus no redraw happens).

Under XFCE sequence is a bit less clear - frame is set to invisible after UnmapNotify event, but then it is set back to visible after second PropertyNotify event (comment near says "Gnome shell does not iconify us when C-z is pressed."). So Emacs thinks that the frame is visible even though it is on different workspace entirely.



In GNU Emacs 28.0.50 (build 19, x86_64-pc-linux-gnu, GTK+ Version 3.24.29, cairo version 1.17.4)
  of 2021-05-14 built on the-big-maker
Repository revision: 203ee33980571823147122dffb0789f92ea66b53
Repository branch: HEAD
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Arch Linux

Configured using:
  'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
  --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
  --with-sound=alsa --with-modules --without-gconf --without-gsettings
  --with-x-toolkit=gtk3 --without-xaw3d'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM HARFBUZZ JPEG JSON LCMS2
LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY PDUMPER PNG
RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM
GTK3 ZLIB

Important settings:
   value of $LC_TIME: en_SE.UTF-8
   value of $LANG: en_US.UTF-8
   locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
   tooltip-mode: t
   global-eldoc-mode: t
   eldoc-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils thingatpt
help-mode cl-loaddefs cl-lib iso-transl tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting font-render-setting cairo move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 53379 13368)
  (symbols 48 6717 0)
  (strings 32 18827 2036)
  (string-bytes 1 618130)
  (vectors 16 13388)
  (vector-slots 8 176246 11054)
  (floats 8 22 47)
  (intervals 56 411 4)
  (buffers 992 12))

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[martin rudalics]

Thanks for the forensics.  They clarify things with Bug#48268.

 > Specifically the commenting of SET_FRAME_VISIBLE(f, 1) causes the problem (if I uncomment that line, buffer redisplay starts to work correctly).
 >
 > Trying the same with XFCE doesn't give the error. I compared the sequences of X events received under different WMs, here's the lists (without "other", "MotionNotify" and key events, for brevity):
 >
 > Xmonad:
 > 1. When switching to different workspace:
 >     - FocusOut x3
 >     - UnmapNotify x1
 >     - LeaveNotify x2
 >     - PropertyNotify x2
 > 2. When switching back:
 >     - MapNotify x1

Can you tell me why we do not SET_FRAME_VISIBLE when receiving the
MapNotify event here?  Probably because we are not yet visible - though
mapped but invisible is a queer state.

 >     - VisibilityNotify x3

Maybe we should process VisibilityNotify events.

 >     - Expose x4

Relying on these for setting visibility will fail for anyone who doesn't
send us an Expose event.

 >     - EnterNotify x2
 >     - FocusIn x2
 >     - ClientMessage x1
 >     - PropertyNotify x1
 >     - FocusIn x1
 >
 > XFCE:
 > 1. When switching to different workspace:
 >     - FocusOut x2
 >     - FocusIn x1
 >     - FocusOut x2
 >     - LeaveNotify x2
 >     - UnmapNotify x1
 >     - PropertyNotify x3
 > 2. When switching back:
 >     - VisibilityNotify x3
 >     - Expose x4
 >     - EnterNotify x2
 >     - PropertyNotify x2
 >     - FocusIn x2
 >     - ClientMessage x1
 >     - PropertyNotify x2
 >     - FocusOut x2
 >     - FocusIn x1
 >     - FocusOut x2
 >     - FocusIn x2

So here we don't get a MapNotify anyway when switching back and using
VisibilityNotify should be TRT.

martin

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

 > So here we don't get a MapNotify anyway when switching back and using
 > VisibilityNotify should be TRT.

Can you please try the attached patch and tell me whether it improves or
breaks things.

Thank you, martin

[-- Attachment #2: VisibilityNotify.diff --]
[-- Type: text/x-patch, Size: 509 bytes --]

diff --git a/src/xterm.c b/src/xterm.c
index 8079a360cf..4a892b3c3e 100644
--- a/src/xterm.c
+++ b/src/xterm.c
@@ -9365,6 +9351,11 @@ handle_one_xevent (struct x_display_info *dpyinfo,
       goto OTHER;

     case VisibilityNotify:
+      f = x_top_window_to_frame (dpyinfo, event->xvisibility.window);
+      if (f && (event->xvisibility.state == VisibilityUnobscured
+		|| event->xvisibility.state == VisibilityPartiallyObscured))
+	SET_FRAME_VISIBLE (f, 1);
+
       goto OTHER;

     case MappingNotify:

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Platon Pronko]

> Can you please try the attached patch and tell me whether it improves or
> breaks things.

Yes, it fixes things - frame is repainted correctly now.

My attempt was similar, but I included more code alongside SET_FRAME_VISIBLE (same as in Expose handler):

+              block_input ();
+             SET_FRAME_VISIBLE (f, 1);
+             if (FRAME_X_DOUBLE_BUFFERED_P (f))
+                font_drop_xrender_surfaces (f);
+              f->output_data.x->has_been_visible = true;
+              SET_FRAME_GARBAGED (f);
+              unblock_input ();

I'm not very familiar with Emacs frame internals - we don't need all these here?

> Relying on these for setting visibility will fail for anyone who doesn't
> send us an Expose event. 

I agree, feels a little fragile.

Best regards,
Platon Pronko

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[martin rudalics]

 >> Can you please try the attached patch and tell me whether it improves or
 >> breaks things.
 >
 > Yes, it fixes things - frame is repainted correctly now.
 >
 > My attempt was similar, but I included more code alongside SET_FRAME_VISIBLE (same as in Expose handler):
 >
 > +              block_input ();
 > +             SET_FRAME_VISIBLE (f, 1);
 > +             if (FRAME_X_DOUBLE_BUFFERED_P (f))
 > +                font_drop_xrender_surfaces (f);
 > +              f->output_data.x->has_been_visible = true;
 > +              SET_FRAME_GARBAGED (f);
 > +              unblock_input ();
 >
 > I'm not very familiar with Emacs frame internals - we don't need all these here?

If I only knew.  If you come up with a patch that works OK for you and
does not break the

(defvar frame (make-frame))
(make-frame-invisible frame)
(frame-visible-p frame)

scenario (where the latter returns 'icon) I want to fix here, I'll
happily install that.

 >> Relying on these for setting visibility will fail for anyone who doesn't
 >> send us an Expose event.
 >
 > I agree, feels a little fragile.

It apparently has not bitten us so far but the Xlib manual is not very
clear in this regard.

martin

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Platon Pronko]

I propose that we include your patch as is (since it keeps the behaviour you need and fixes the issues - hopefully Basil's issue also goes away).

All the SET_FRAME_GARBAGED and has_been_visible manipulations don't seem to be immediately relevant - redrawing works fine without it. And we shouldn't just copy-paste code with unknown purpose :)

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[martin rudalics]

 > I propose that we include your patch as is (since it keeps the
 > behaviour you need and fixes the issues - hopefully Basil's issue also
 > goes away).

I pushed it now.

 > All the SET_FRAME_GARBAGED and has_been_visible manipulations don't
 > seem to be immediately relevant - redrawing works fine without it. And
 > we shouldn't just copy-paste code with unknown purpose :)

Please keep your additional changes ready so we can apply them as soon
as we find more problems in this area.

Thanks again for the analytics, martin

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Platon Pronko]

Thank you! I tested the latest master, it works fine. This bug can be closed now.

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[martin rudalics]

 > Thank you! I tested the latest master, it works fine. This bug can be closed now.

Let's wait for a day at least.

BTW, you didn't answer one of my earlier questions namely this one:

 >  > Xmonad:
 >  > 1. When switching to different workspace:
 >  >     - FocusOut x3
 >  >     - UnmapNotify x1
 >  >     - LeaveNotify x2
 >  >     - PropertyNotify x2
 >  > 2. When switching back:
 >  >     - MapNotify x1
 >
 > Can you tell me why we do not SET_FRAME_VISIBLE when receiving the
 > MapNotify event here?  Probably because we are not yet visible - though
 > mapped but invisible is a queer state.

In Bug#48129 Tom conjectures that this happens because some WMs do not
set _NET_WM_STATE.  Do you agree with him?  If so, wouldn't it make
sense to skip that x_get_current_wm_state check in MapNotify because it
fails on too many WMs?

martin

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Platon Pronko]

>> Can you tell me why we do not SET_FRAME_VISIBLE when receiving the
>> MapNotify event here?  Probably because we are not yet visible - though
>> mapped but invisible is a queer state.
> 
> In Bug#48129 Tom conjectures that this happens because some WMs do not
> set _NET_WM_STATE.  Do you agree with him?  If so, wouldn't it make
> sense to skip that x_get_current_wm_state check in MapNotify because it
> fails on too many WMs? 

I think I agree with your reasoning ("Probably because we are not yet visible"). Can't comment further because I don't know much about Emacs frame internals and different WM specifics.

Best regards,
Platon Pronko

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Stefan Kangas]

Platon Pronko <platon7pronko@gmail.com> writes:

>>> Can you tell me why we do not SET_FRAME_VISIBLE when receiving the
>>> MapNotify event here?  Probably because we are not yet visible - though
>>> mapped but invisible is a queer state.
>> In Bug#48129 Tom conjectures that this happens because some WMs do not
>> set _NET_WM_STATE.  Do you agree with him?  If so, wouldn't it make
>> sense to skip that x_get_current_wm_state check in MapNotify because it
>> fails on too many WMs?
>
> I think I agree with your reasoning ("Probably because we are not yet
> visible"). Can't comment further because I don't know much about Emacs
> frame internals and different WM specifics.

Is there anything more to do here, or should this bug be closed?

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Platon Pronko]

On 2021-09-29 15:07, Stefan Kangas wrote:
> 
> Is there anything more to do here, or should this bug be closed?

I (as an author of the bug) think this bug should be closed, since the original issue is fully resolved.

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Stefan Kangas]

tags 48413 fixed
close 48413 28.1
thanks

Platon Pronko <platon7pronko@gmail.com> writes:

> On 2021-09-29 15:07, Stefan Kangas wrote:
>>
>> Is there anything more to do here, or should this bug be closed?
>
> I (as an author of the bug) think this bug should be closed, since the original issue is fully resolved.

Thanks, I'm therefore closing this bug.

Martin, if this is wrong, please reopen.

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[martin rudalics]

 > Thanks, I'm therefore closing this bug.
 >
 > Martin, if this is wrong, please reopen.

It's OK with me.

Thanks, martin

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Basil L. Contovounesios]

martin rudalics <rudalics@gmx.at> writes:

>> So here we don't get a MapNotify anyway when switching back and using
>> VisibilityNotify should be TRT.
>
> Can you please try the attached patch and tell me whether it improves or
> breaks things.

Seems to fix all the issues I saw on Lucid, GTK2, and GTK3, FWIW.

Thanks,

-- 
Basil

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[martin rudalics]

 >> Can you please try the attached patch and tell me whether it improves or
 >> breaks things.
 >
 > Seems to fix all the issues I saw on Lucid, GTK2, and GTK3, FWIW.

I installed it now.

Thanks for testing, martin

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Basil L. Contovounesios]

Platon Pronko <platon7pronko@gmail.com> writes:

> This looks like Xmonad-specific regression after commit 483c5e95 by Martin Rudalics.
>
> Steps to reproduce (under Xmonad):
>
> 1. Open a new frame (./src/emacs -Q);
> 2. Type something in a buffer;
> 3. Switch to a different workspace;
> 4. Switch back;
> 5. Observe buffer being blank and not responding to any interaction.

FWIW, on Xmonad 0.15 when I switch back the selected frame is not blank.
Only when there is more than one frame are the unselected frames blank.
(See screenshots in https://bugs.gnu.org/48268#20.)

Thanks,

-- 
Basil

bug#48413: 28.0.50; emacs does not repaint the window after switching workspaces in Xmonad

[Basil L. Contovounesios]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> Platon Pronko <platon7pronko@gmail.com> writes:
>
>> This looks like Xmonad-specific regression after commit 483c5e95 by Martin Rudalics.
>>
>> Steps to reproduce (under Xmonad):
>>
>> 1. Open a new frame (./src/emacs -Q);
>> 2. Type something in a buffer;
>> 3. Switch to a different workspace;
>> 4. Switch back;
>> 5. Observe buffer being blank and not responding to any interaction.
>
> FWIW, on Xmonad 0.15 when I switch back the selected frame is not blank.
> Only when there is more than one frame are the unselected frames blank.
> (See screenshots in https://bugs.gnu.org/48268#20.)

Ah, but this is true for me only on Lucid.  With GTK2 and GTK3 I can
indeed reproduce the blanking.  FWIW, the blank frame responds to
'C-x 5 2', creating a second frame that is not blank.

-- 
Basil