bug#19721: 25.0.50; Mode-line not redrawn with expose events

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[YAMAMOTO Mitsuharu]

[-- Attachment #1: Type: text/plain, Size: 1493 bytes --]

This bug is related to the mode-line erasure problem I mentioned in
http://lists.gnu.org/archive/html/emacs-devel/2015-01/msg01040.html .

To reproduce the problem by the instruction below, you would need to
make sure that your X11 compositing manager is turned off.  I tested
on Cent OS 5.11, default setting.  See also
http://lists.gnu.org/archive/html/emacs-devel/2013-04/msg00600.html .

Steps to reproduce:

1. Create a file (say, ~/test.el) containing the following contents:

  (custom-set-faces
   '(mode-line ((((class color) (min-colors 88)) (:background
     "grey75" :foreground "black" :box (:line-width 2 :color
     "grey75" :style released-button)))))
   )

2. $ emacs -Q -D -l ~/test.el
3. C-x 2 C-x 2 C-x 2
4. C-x o C-x o C-x o
5. Move another window (e.g., the terminal window from which Emacs is
   invoked) so that it hovers over the Emacs frame.

Result:

The upper two mode-lines among four are not redrawn after their hidden
part is revealed (see the attachment).  They are not redrawn in
response to expose events because the flag `enabled_p' for these
mode-line glyph rows have been set to false.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

In GNU Emacs 25.0.50.1 (i686-pc-linux-gnu, GTK+ Version 2.10.4)
 of 2015-01-29 on localhost.localdomain
Windowing system distributor `The X.Org Foundation', version 11.0.70101000
System Description:	CentOS release 5.11 (Final)

Configured features:
XPM JPEG TIFF GIF PNG SOUND LIBSELINUX FREETYPE XFT ZLIB

[-- Attachment #2: mode-line-erasure.png --]
[-- Type: image/png, Size: 41074 bytes --]

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[YAMAMOTO Mitsuharu]

I could reproduce the bug with Emacs 24.4 but not with Emacs 24.3.

I also tried git bisect.  The result was:

655ab9a380068143cfb9a31d01488e83676d81c1 is the first bad commit
commit 655ab9a380068143cfb9a31d01488e83676d81c1
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date:   Thu Nov 28 17:43:09 2013 -0500

    Refine redisplay optimizations to only redisplay *some* frames/windows
    rather than all of them.
    * src/xdisp.c (REDISPLAY_SOME): New constant.
    (redisplay_other_windows, wset_redisplay, fset_redisplay)
    (bset_redisplay, bset_update_mode_line): New functions.
    (message_dolog): Use bset_redisplay.
    (clear_garbaged_frames): Use fset_redisplay.
    (echo_area_display): Use wset_redisplay.
    (buffer_shared_and_changed): Remove.
    (prepare_menu_bars): Call Vpre_redisplay_function before updating
    frame titles.  Compute the actual set of windows redisplayed.
    Don't update frame titles and menu bars for frames that don't need to
    be redisplayed.
    (propagate_buffer_redisplay): New function.
    (AINC): New macro.
    (redisplay_internal): Use it.  Be more selective in the set of windows
    we redisplay.  Propagate windows_or_buffers_changed to
    update_mode_lines a bit later to simplify the code.
    (mark_window_display_accurate_1): Reset window and buffer's
    `redisplay' flag.
    (redisplay_window): Do nothing if neither the window nor the buffer nor
    the frame needs redisplay.
    * src/window.h (struct window): Add `redisplay' field.
    (wset_redisplay, fset_redisplay, bset_redisplay, bset_update_mode_line)
    (redisplay_other_windows, window_list): New declarations.
    * src/window.c (select_window, Fset_window_start): Use wset_redisplay.
    (window_list): Not static any more.
    (grow_mini_window, shrink_mini_window): Use fset_redisplay.
    * src/minibuf.c (read_minibuf_unwind): Don't redisplay everything.
    * src/insdel.c (prepare_to_modify_buffer_1): Use bset_redisplay.
    * src/frame.c (Fmake_frame_visible): Don't redisplay everything.
    * src/frame.h (struct frame): Add `redisplay' field.
    Move `external_menu_bar' bitfield next to other bit-fields.
    (SET_FRAME_GARBAGED): Use fset_redisplay.
    (SET_FRAME_VISIBLE): Don't garbage the frame;
    Use redisplay_other_windows.
    * src/buffer.h (struct buffer): Add `redisplay' field.
    * src/buffer.c (Fforce_mode_line_update): Pay attention to the `all' flag.
    (modify_overlay): Use bset_redisplay.
    * src/alloc.c (gc_sweep): Don't unmark strings while sweeping symbols.
    
    * lisp/doc-view.el (doc-view-goto-page): Update mode-line.

:040000 040000 539134fa8301dbfe574c24f72a568daf322768b2 6e5b613809d7c5dcf21ac1b3be5f21ef4e3d877b M	lisp
:040000 040000 518107ab34d68fedf97e35cfac6b8a7e78f9c67c 5acb1b218288451c64bae59b039d359a046aec78 M	src

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

>>>>> On Thu, 29 Jan 2015 19:51:24 +0900, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> said:

> This bug is related to the mode-line erasure problem I mentioned in
> http://lists.gnu.org/archive/html/emacs-devel/2015-01/msg01040.html .

> To reproduce the problem by the instruction below, you would need to
> make sure that your X11 compositing manager is turned off.  I tested
> on Cent OS 5.11, default setting.  See also
> http://lists.gnu.org/archive/html/emacs-devel/2013-04/msg00600.html .

> Steps to reproduce:

> 1. Create a file (say, ~/test.el) containing the following contents:

>   (custom-set-faces
>    '(mode-line ((((class color) (min-colors 88)) (:background
>      "grey75" :foreground "black" :box (:line-width 2 :color
>      "grey75" :style released-button)))))
>    )

> 2. $ emacs -Q -D -l ~/test.el
> 3. C-x 2 C-x 2 C-x 2
> 4. C-x o C-x o C-x o
> 5. Move another window (e.g., the terminal window from which Emacs is
>    invoked) so that it hovers over the Emacs frame.

> Result:

> The upper two mode-lines among four are not redrawn after their hidden
> part is revealed (see the attachment).  They are not redrawn in
> response to expose events because the flag `enabled_p' for these
> mode-line glyph rows have been set to false.

> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp

> In GNU Emacs 25.0.50.1 (i686-pc-linux-gnu, GTK+ Version 2.10.4)
>  of 2015-01-29 on localhost.localdomain
> Windowing system distributor `The X.Org Foundation', version 11.0.70101000
> System Description:	CentOS release 5.11 (Final)

> Configured features:
> XPM JPEG TIFF GIF PNG SOUND LIBSELINUX FREETYPE XFT ZLIB

 

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Sat, 31 Jan 2015 19:31:59 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> 
> I could reproduce the bug with Emacs 24.4 but not with Emacs 24.3.
> 
> I also tried git bisect.  The result was:
> 
> 655ab9a380068143cfb9a31d01488e83676d81c1 is the first bad commit
> commit 655ab9a380068143cfb9a31d01488e83676d81c1
> Author: Stefan Monnier <monnier@iro.umontreal.ca>
> Date:   Thu Nov 28 17:43:09 2013 -0500
> 
>     Refine redisplay optimizations to only redisplay *some* frames/windows
>     rather than all of them.

Thanks.

Yes, it was quite clear to me that this was caused by that change.
But the question is how to fix this regression without going back to
all the unnecessary redrawing which that change removed.

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Thu, 29 Jan 2015 19:51:24 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> 
> This bug is related to the mode-line erasure problem I mentioned in
> http://lists.gnu.org/archive/html/emacs-devel/2015-01/msg01040.html .
> 
> To reproduce the problem by the instruction below, you would need to
> make sure that your X11 compositing manager is turned off.  I tested
> on Cent OS 5.11, default setting.  See also
> http://lists.gnu.org/archive/html/emacs-devel/2013-04/msg00600.html .
> 
> Steps to reproduce:
> 
> 1. Create a file (say, ~/test.el) containing the following contents:
> 
>   (custom-set-faces
>    '(mode-line ((((class color) (min-colors 88)) (:background
>      "grey75" :foreground "black" :box (:line-width 2 :color
>      "grey75" :style released-button)))))
>    )
> 
> 2. $ emacs -Q -D -l ~/test.el
> 3. C-x 2 C-x 2 C-x 2
> 4. C-x o C-x o C-x o
> 5. Move another window (e.g., the terminal window from which Emacs is
>    invoked) so that it hovers over the Emacs frame.
> 
> Result:
> 
> The upper two mode-lines among four are not redrawn after their hidden
> part is revealed (see the attachment).  They are not redrawn in
> response to expose events because the flag `enabled_p' for these
> mode-line glyph rows have been set to false.

Does the patch below fix the problem?

I cannot reproduce your recipe on the systems to which I have access,
so I used a slightly modified procedure and the dump-glyph-matrix
function to show what I think is the same issue: namely, in a frame
with 3 vertically-arranged windows, the window "after the selected
one" (in the next-window sense) has its mode-line row disabled in the
current glyph matrix.  That problem is solved by the patch below, but
I'm not sure it solves yours.

Thanks.

diff --git a/src/dispnew.c b/src/dispnew.c
index f73ea58..0b375dc 100644
--- a/src/dispnew.c
+++ b/src/dispnew.c
@@ -562,7 +562,8 @@ struct redisplay_history
 	      if (w->window_end_vpos >= i)
 		w->window_end_valid = 0;
 
-	      while (i < matrix->nrows)
+	      int nrows = matrix->nrows - WINDOW_WANTS_MODELINE_P (w);
+	      while (i < nrows)
 		matrix->rows[i++].enabled_p = false;
 	    }
 	  else

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[YAMAMOTO Mitsuharu]

[-- Attachment #1: Type: text/plain, Size: 1039 bytes --]

>>>>> On Sat, 31 Jan 2015 12:53:37 +0200, Eli Zaretskii <eliz@gnu.org> said:

> Does the patch below fix the problem?

> I cannot reproduce your recipe on the systems to which I have access,
> so I used a slightly modified procedure and the dump-glyph-matrix
> function to show what I think is the same issue: namely, in a frame
> with 3 vertically-arranged windows, the window "after the selected
> one" (in the next-window sense) has its mode-line row disabled in the
> current glyph matrix.  That problem is solved by the patch below, but
> I'm not sure it solves yours.

With this patch, the mode-lines are redrawn when they are exposed.
But I also observe that a mode-line part is not updated with the last
C-x o (see the 3rd mode-line in the screenshot) with the previously
mentioned procedure on the Mac port.  Strangely, if I try C-x o three
times further, then the second mode-line part is also not updated.
Unfortunately, I couldn't reproduce it with Cent OS 5.11.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp


[-- Attachment #2: mode-line-not-update.png --]
[-- Type: image/png, Size: 72528 bytes --]

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Sat, 31 Jan 2015 20:37:09 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> Cc: 	19721@debbugs.gnu.org
> 
> With this patch, the mode-lines are redrawn when they are exposed.
> But I also observe that a mode-line part is not updated with the last
> C-x o (see the 3rd mode-line in the screenshot) with the previously
> mentioned procedure on the Mac port.  Strangely, if I try C-x o three
> times further, then the second mode-line part is also not updated.

Right.  Which means it is not enough to avoid disabling the mode line,
we must actually force its redrawing afterwards.

How about the following patch instead of the previous one?

diff --git a/src/dispnew.c b/src/dispnew.c
index f73ea58..de9791e 100644
--- a/src/dispnew.c
+++ b/src/dispnew.c
@@ -564,6 +564,8 @@ struct redisplay_history
 
 	      while (i < matrix->nrows)
 		matrix->rows[i++].enabled_p = false;
+	      if (WINDOW_WANTS_MODELINE_P (w))
+		w->update_mode_line = 1;
 	    }
 	  else
 	    {

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 31 Jan 2015 13:44:38 +0200, Eli Zaretskii <eliz@gnu.org> said:

>> With this patch, the mode-lines are redrawn when they are exposed.
>> But I also observe that a mode-line part is not updated with the
>> last C-x o (see the 3rd mode-line in the screenshot) with the
>> previously mentioned procedure on the Mac port.  Strangely, if I
>> try C-x o three times further, then the second mode-line part is
>> also not updated.

> Right.  Which means it is not enough to avoid disabling the mode
> line, we must actually force its redrawing afterwards.

> How about the following patch instead of the previous one?

Now mode-lines get updated with C-x o again, but the original issue
still remains and I could reproduce it with the original procedure on
Cent OS 5.11.

If I apply both changes, mode-lines are redrawn when exposed, but some
of them are not updated with C-x o.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[martin rudalics]

 > If I apply both changes, mode-lines are redrawn when exposed, but some
 > of them are not updated with C-x o.

What happens when you do something trivial like

--- a/src/window.c
+++ b/src/window.c
@@ -524,6 +524,8 @@ select_window (Lisp_Object window, Lisp_Object norecord, int inhibit_point_swap)
        record_buffer (w->contents);
      }

+  w->update_mode_line = 1;
+
    return window;
  }

martin

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 09:51:30 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 19721@debbugs.gnu.org
> 
>  > If I apply both changes, mode-lines are redrawn when exposed, but some
>  > of them are not updated with C-x o.
> 
> What happens when you do something trivial like
> 
> --- a/src/window.c
> +++ b/src/window.c
> @@ -524,6 +524,8 @@ select_window (Lisp_Object window, Lisp_Object norecord, int inhibit_point_swap)
>         record_buffer (w->contents);
>       }
> 
> +  w->update_mode_line = 1;
> +
>     return window;
>   }

Thanks.

I didn't try your suggestion, but (a) it would force redrawing the
mode line even if that's not needed, just because a window got
selected, and (b) there's a tricky condition near the beginning of
redisplay_window that would bypass redisplaying a window, under some
conditions, even if its update_mode_line flag was set (my last patch
attempts at fixing that).

So I'm not sure this is the right solution.  The situation described
in this report is quite unique, in that the face used for the active
mode line causes the window glyph matrices to be resized each time a
window becomes non-selected one.  It is because of this resizing that
the mode-line row of the current matrix becomes disabled.

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[martin rudalics]

 > I didn't try your suggestion, but (a) it would force redrawing the
 > mode line even if that's not needed, just because a window got
 > selected,

Yes.  We could be a bit more careful and do it only when norecord is
nil.  My point was just to know whether the bug would disappear.

 > and (b) there's a tricky condition near the beginning of
 > redisplay_window that would bypass redisplaying a window, under some
 > conditions, even if its update_mode_line flag was set (my last patch
 > attempts at fixing that).

So if he applies your code and mine we'd probably find out more.  I was
exactly once able to trigger his initial scenario here on Windows with
both upper modelines completely disappearing but was not able to repeat
that experience after that.

 > So I'm not sure this is the right solution.  The situation described
 > in this report is quite unique, in that the face used for the active
 > mode line causes the window glyph matrices to be resized each time a
 > window becomes non-selected one.

Could you optimize that away (reserving one line more than needed)?
A naive question, probably ...

 > It is because of this resizing that
 > the mode-line row of the current matrix becomes disabled.

So you mean that we have to update the mode lines of both - the selected
and the deselected window?

martin

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 17:30:06 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: mituharu@math.s.chiba-u.ac.jp, 19721@debbugs.gnu.org
> 
>  > and (b) there's a tricky condition near the beginning of
>  > redisplay_window that would bypass redisplaying a window, under some
>  > conditions, even if its update_mode_line flag was set (my last patch
>  > attempts at fixing that).
> 
> So if he applies your code and mine we'd probably find out more.  I was
> exactly once able to trigger his initial scenario here on Windows with
> both upper modelines completely disappearing but was not able to repeat
> that experience after that.

I couldn't reproduce the problem with redrawing on expose events
(which is not surprising, since on Windows expose events are almost
never used, except when the frame is first displayed after it's
created).  But I have no trouble at all seeing that the mode-line
glyph row is disabled, by using dump-glyph-matrix.

>  > So I'm not sure this is the right solution.  The situation described
>  > in this report is quite unique, in that the face used for the active
>  > mode line causes the window glyph matrices to be resized each time a
>  > window becomes non-selected one.
> 
> Could you optimize that away (reserving one line more than needed)?

That was my first attempt, and it didn't do a thorough enough job, as
you can see from previous discussions.

>  > It is because of this resizing that
>  > the mode-line row of the current matrix becomes disabled.
> 
> So you mean that we have to update the mode lines of both - the selected
> and the deselected window?

Yes, but not because they become selected/deselected, because their
glyph matrices are resized, and the mode-line row gets disabled in the
process.

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[martin rudalics]

 >> So you mean that we have to update the mode lines of both - the selected
 >> and the deselected window?
 >
 > Yes, but not because they become selected/deselected, because their
 > glyph matrices are resized, and the mode-line row gets disabled in the
 > process.

But the glyph matrices are resized _because_ the window gets
(de-)selected?  In the OP's case, obviously.

martin

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 19:09:35 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: mituharu@math.s.chiba-u.ac.jp, 19721@debbugs.gnu.org
> 
>  >> So you mean that we have to update the mode lines of both - the selected
>  >> and the deselected window?
>  >
>  > Yes, but not because they become selected/deselected, because their
>  > glyph matrices are resized, and the mode-line row gets disabled in the
>  > process.
> 
> But the glyph matrices are resized _because_ the window gets
> (de-)selected?

Yes, in this use case.  Not sure why this is significant, though: we
should redraw the mode line when the matrix is resized, no matter what
triggered the resize.

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 01 Feb 2015 09:51:30 +0100, martin rudalics <rudalics@gmx.at> said:

>> If I apply both changes, mode-lines are redrawn when exposed, but some
>> of them are not updated with C-x o.

> What happens when you do something trivial like

> --- a/src/window.c
> +++ b/src/window.c
> @@ -524,6 +524,8 @@ select_window (Lisp_Object window, Lisp_Object norecord, int inhibit_point_swap)
>         record_buffer (w->contents);
>       }

> +  w->update_mode_line = 1;
> +
>     return window;
>   }

The result was similar to Eli's second patch.  With your patch alone,
the original problem remains.  With Eli's first patch, some of
mode-lines are not update with C-x o.

>>>>> On Sun, 01 Feb 2015 17:44:01 +0200, Eli Zaretskii <eliz@gnu.org> said:

> So I'm not sure this is the right solution.  The situation described
> in this report is quite unique, in that the face used for the active
> mode line causes the window glyph matrices to be resized each time a
> window becomes non-selected one.  It is because of this resizing that
> the mode-line row of the current matrix becomes disabled.

I've experienced similar mode-line erasure without customization of
the mode-line face on the Mac port a few times (probably after
pixel-based mouse-wheel smooth scrolling over an inactive window).
But I couldn't find a way to reproducible it reliably.  The face
customization example was originally given by a user of the Mac port
as a part of a bug report I received.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 14:40:40 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> Cc: 19721@debbugs.gnu.org
> 
> > How about the following patch instead of the previous one?
> 
> Now mode-lines get updated with C-x o again, but the original issue
> still remains and I could reproduce it with the original procedure on
> Cent OS 5.11.
> 
> If I apply both changes, mode-lines are redrawn when exposed, but some
> of them are not updated with C-x o.

How about the patch below?  (Once again, it's wrt the current emacs-24
branch.)

diff --git a/src/dispnew.c b/src/dispnew.c
index f73ea58..6517c9b 100644
--- a/src/dispnew.c
+++ b/src/dispnew.c
@@ -570,6 +570,12 @@ struct redisplay_history
 	      for (i = 0; i < matrix->nrows; ++i)
 		matrix->rows[i].enabled_p = false;
 	    }
+	  /* We've disabled the mode-line row, so force redrawing of
+	     the mode line, if any, since otherwise it will remain
+	     disabled in the current matrix, and expose events won't
+	     redraw it.  */
+	  if (WINDOW_WANTS_MODELINE_P (w))
+	    w->update_mode_line = 1;
 	}
       else if (matrix == w->desired_matrix)
 	{
diff --git a/src/xdisp.c b/src/xdisp.c
index b1125d3..2ebf06d 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -15964,6 +15964,7 @@ enum
   if (!just_this_one_p
       && REDISPLAY_SOME_P ()
       && !w->redisplay
+      && !w->update_mode_line
       && !f->redisplay
       && !buffer->text->redisplay
       && BUF_PT (buffer) == w->last_point)

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 01 Feb 2015 17:33:10 +0200, Eli Zaretskii <eliz@gnu.org> said:

> How about the patch below?  (Once again, it's wrt the current emacs-24
> branch.)

This works fine on my side.  Thanks.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

bug#19721: 25.0.50; Mode-line not redrawn with expose events

[Eli Zaretskii]

> Date: Mon, 02 Feb 2015 12:27:39 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> Cc: 19721@debbugs.gnu.org
> 
> >>>>> On Sun, 01 Feb 2015 17:33:10 +0200, Eli Zaretskii <eliz@gnu.org> said:
> 
> > How about the patch below?  (Once again, it's wrt the current emacs-24
> > branch.)
> 
> This works fine on my side.  Thanks.

Thanks, I pushed this as commit e9a7e10 to the emacs-24 branch.