Redisplay problems?

Redisplay problems?

[Christian Lynbech]

Are there any known redisplay issues or is the redisplay machinery
undergoing some transformations currently?

I am using the latest version of emacs 24 from the repository, I have
just updated and rebuilt today and I am seeing some strange and rather
annoying problems.

I will try to explain what I am seeing, let me know if it is to
incoherent to make sense, I can lok into creating a video or something.

My problem essential is that I seem to lose the cursor upon startup (as
it becoming invisible, I still have a welldefined point where
charactyers are inserted, but I cannot see the cursor itself). I have a
rather involved .emacs, built up over many years, using a multiframe
setup. Basically, during startup I load and configure a bunch of stuff
and open a couple of predefined frames which are closed again, leaving
just one frame open. When .emacs evaluation is finshed and I look at my
open frame, it has no cursor. Typing C-l doesn't help, the cursor
remains invisible. If I then switch to one of my other frames, the
cursor reappears in the initial frame, but if I continue to de-iconify
other frames, they too has an invisble cursor. Switching between now no
longer helps, so I need to delete those frames and recreate them. Once
they have a visible cursor, it seems to stay visible. Starting up with
-q (or --daemon) seems to avert the issue, so I do not have any easy way
of reproducing it.

My theory is that something topples due to these frames being creating
during initialization (.emacs evaluation) and C-l for some reason either
doesn't work as reliably as it once did or something optimizes away some
redisplay that I need.

I am unsure exactly when this behaviour has started, sometime last year
(2013). At first I thought it was just related to me running on a rather
old RedHat Linux server installation, but now all my installations on
both Linux and Mac OSX are affected. I do not use gnome and compile with
the lucid toolkit.

Does this ring a bell with anyone?

I think I shall be able to rewrite my startup to code around the issue
(at least if the time of creating the frames somehow is involved) but I
wanted to raise the issue before just burying it. I know the above
description is in itself not usefull for debugging the problem, let me
know if there is any other information I can provide or if you want to
pursue it at all.


------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Redisplay problems?

[Eli Zaretskii]

> From: Christian Lynbech <christian@defun.dk>
> Date: Mon, 10 Mar 2014 21:37:10 +0100
> 
> Are there any known redisplay issues or is the redisplay machinery
> undergoing some transformations currently?

Changes in the display engine are being committed all the time.

> My problem essential is that I seem to lose the cursor upon startup (as
> it becoming invisible, I still have a welldefined point where
> charactyers are inserted, but I cannot see the cursor itself). I have a
> rather involved .emacs, built up over many years, using a multiframe
> setup. Basically, during startup I load and configure a bunch of stuff
> and open a couple of predefined frames which are closed again, leaving
> just one frame open. When .emacs evaluation is finshed and I look at my
> open frame, it has no cursor. Typing C-l doesn't help, the cursor
> remains invisible. If I then switch to one of my other frames, the
> cursor reappears in the initial frame, but if I continue to de-iconify
> other frames, they too has an invisble cursor. Switching between now no
> longer helps, so I need to delete those frames and recreate them. Once
> they have a visible cursor, it seems to stay visible. Starting up with
> -q (or --daemon) seems to avert the issue, so I do not have any easy way
> of reproducing it.

Sounds like some weird side effect of the changes in blinking cursor
feature.  Do you customize any of the related options?

Re: Redisplay problems?

[James Cloos]

I've seen the lost cursor, too.  But cannot reliably reproduce.

It seems to occur more often if, when gnus is starting up, I switch back
to the workspace which includes the emacs frame too many times.

That is, if emacs gets a lot of these sorts of events while waiting for
something like gnus to finish running:

(Generated here by running xev(1) and switching workspaces from and to.
These are therefore the events my window manager sends when switching
workspaces.)

,----
| PropertyNotify event, serial 38, synthetic NO, window 0x2a00001,
|     atom 0x134 (_WIN_STATE), time 2073529870, state PropertyNewValue
| 
| PropertyNotify event, serial 38, synthetic NO, window 0x2a00001,
|     atom 0x149 (_NET_WM_STATE), time 2073529870, state PropertyNewValue
| 
| PropertyNotify event, serial 38, synthetic NO, window 0x2a00001,
|     atom 0x122 (WM_STATE), time 2073529870, state PropertyNewValue
| 
| UnmapNotify event, serial 38, synthetic NO, window 0x2a00001,
|     event 0x2a00001, window 0x2a00001, from_configure NO
| 
| ColormapNotify event, serial 38, synthetic NO, window 0x2a00001,
|     colormap 0x20, new NO, state ColormapInstalled
| 
| PropertyNotify event, serial 38, synthetic NO, window 0x2a00001,
|     atom 0x134 (_WIN_STATE), time 2073530086, state PropertyNewValue
| 
| PropertyNotify event, serial 38, synthetic NO, window 0x2a00001,
|     atom 0x149 (_NET_WM_STATE), time 2073530086, state PropertyNewValue
| 
| PropertyNotify event, serial 38, synthetic NO, window 0x2a00001,
|     atom 0x122 (WM_STATE), time 2073530086, state PropertyNewValue
| 
| MapNotify event, serial 38, synthetic NO, window 0x2a00001,
|     event 0x2a00001, window 0x2a00001, override NO
| 
| VisibilityNotify event, serial 38, synthetic NO, window 0x2a00001,
|     state VisibilityUnobscured
| 
| Expose event, serial 38, synthetic NO, window 0x2a00001,
|     (0,0), width 178, height 10, count 3
| 
| Expose event, serial 38, synthetic NO, window 0x2a00001,
|     (0,10), width 10, height 58, count 2
| 
| Expose event, serial 38, synthetic NO, window 0x2a00001,
|     (68,10), width 110, height 58, count 1
| 
| Expose event, serial 38, synthetic NO, window 0x2a00001,
|     (0,68), width 178, height 110, count 0
| 
| ColormapNotify event, serial 38, synthetic NO, window 0x2a00001,
|     colormap 0x20, new NO, state ColormapUninstalled
`----

The other symptom is that eventually nothing displays in the frame.

But I cannot duplicate it at will, and I hadn't even confirmed whether
it is an emacs bug or a xlib or xserver bug.  (My xorg is all git master.)

I use --with-x-toolkit=lucid --with-xaw3d.

-JimC
--
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: Redisplay problems?

[Kan-Ru Chen (陳侃如)]

I find a way to reproduce it reliably. Just run

emacs -Q --eval "(progn (scroll-bar-mode -1) \
                        (iconify-or-deiconify-frame) \
                        (sleep-for 1))"

Version (compiled from bzr):
GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18

then deiconify the opened frame. The cursor is gone. The frame does not
redisplay correctly either.

    Kanru

Re: Redisplay problems?

[Eli Zaretskii]

> From: Kan-Ru Chen (陳侃如) <kanru@kanru.info>
> Date: Tue, 18 Mar 2014 18:48:57 +0800
> Cc: Christian Lynbech <christian@defun.dk>, emacs-devel@gnu.org
> 
> I find a way to reproduce it reliably. Just run
> 
> emacs -Q --eval "(progn (scroll-bar-mode -1) \
>                         (iconify-or-deiconify-frame) \
>                         (sleep-for 1))"
> 
> Version (compiled from bzr):
> GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18
> 
> then deiconify the opened frame. The cursor is gone. The frame does not
> redisplay correctly either.

Not reproducible here, but I'm not on GNU/Linux with GTK+.

Re: Redisplay problems?

[Óscar Fuentes]

"Kan-Ru Chen (陳侃如)" <kanru@kanru.info> writes:

> I find a way to reproduce it reliably. Just run
>
> emacs -Q --eval "(progn (scroll-bar-mode -1) \
>                         (iconify-or-deiconify-frame) \
>                         (sleep-for 1))"
>
> Version (compiled from bzr):
> GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18
>
> then deiconify the opened frame. The cursor is gone. The frame does not
> redisplay correctly either.

Reproducible with

GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, X toolkit) of 2014-01-21

Re: Redisplay problems?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 18 Mar 2014 17:17:05 +0100
> 
> "Kan-Ru Chen (陳侃如)" <kanru@kanru.info> writes:
> 
> > I find a way to reproduce it reliably. Just run
> >
> > emacs -Q --eval "(progn (scroll-bar-mode -1) \
> >                         (iconify-or-deiconify-frame) \
> >                         (sleep-for 1))"
> >
> > Version (compiled from bzr):
> > GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18
> >
> > then deiconify the opened frame. The cursor is gone. The frame does not
> > redisplay correctly either.
> 
> Reproducible with
> 
> GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, X toolkit) of 2014-01-21

How about bisecting it?

Re: Redisplay problems?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> "Kan-Ru Chen (陳侃如)" <kanru@kanru.info> writes:
>> 
>> > I find a way to reproduce it reliably. Just run
>> >
>> > emacs -Q --eval "(progn (scroll-bar-mode -1) \
>> >                         (iconify-or-deiconify-frame) \
>> >                         (sleep-for 1))"
>> >
>> > Version (compiled from bzr):
>> > GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18
>> >
>> > then deiconify the opened frame. The cursor is gone. The frame does not
>> > redisplay correctly either.
>> 
>> Reproducible with
>> 
>> GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, X toolkit) of 2014-01-21
>
> How about bisecting it?

I'll give it a try this night if nobody beats me to it.

Re: Redisplay problems?

[Christian Lynbech]

I can also add that I did have turned off cursor blinking. However
outcommenting the code did not really help, possibly it alleviated the
issue on OSX but on Linux it remained the same.

I also have disabled scrollbars in my setup.

                              -- Christian

Re: Redisplay problems?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 18 Mar 2014 17:53:11 +0100
> 
> >> Reproducible with
> >> 
> >> GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, X toolkit) of 2014-01-21
> >
> > How about bisecting it?
> 
> I'll give it a try this night if nobody beats me to it.

Thanks.

Re: Redisplay problems?

[James Cloos]

>>>>> "KC" == Kan-Ru Chen (陳侃如) <kanru@kanru.info> writes:

KC> I find a way to reproduce it reliably. Just run
KC> emacs -Q --eval "(progn (scroll-bar-mode -1) \
KC>                         (iconify-or-deiconify-frame) \
KC>                         (sleep-for 1))"

KC> Version (compiled from bzr):
KC> GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18

KC> then deiconify the opened frame. The cursor is gone. The frame does not
KC> redisplay correctly either.

Yes, that also reproduces it here, with athena, also from bzr, albeit a
bit older:

GNU Emacs 24.3.50.1 (x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2014-03-11

-JimC
--
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: Redisplay problems?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 18 Mar 2014 17:17:05 +0100
>> 
>> "Kan-Ru Chen (陳侃如)" <kanru@kanru.info> writes:
>> 
>> > I find a way to reproduce it reliably. Just run
>> >
>> > emacs -Q --eval "(progn (scroll-bar-mode -1) \
>> >                         (iconify-or-deiconify-frame) \
>> >                         (sleep-for 1))"
>> >
>> > Version (compiled from bzr):
>> > GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18
>> >
>> > then deiconify the opened frame. The cursor is gone. The frame does not
>> > redisplay correctly either.
>> 
>> Reproducible with
>> 
>> GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, X toolkit) of 2014-01-21
>
> How about bisecting it?

241fd4407576cb9a417a0f0a546a164cf56c927a is the first bad commit
commit 241fd4407576cb9a417a0f0a546a164cf56c927a
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date:   Thu Nov 28 17:43:09 2013 -0500

    Refine redisplay optimizations to only redisplay *some* frames/windows
    rather than all of them.
    * src/xdisp.c (REDISPLAY_SOME): New constant.
    (redisplay_other_windows, wset_redisplay, fset_redisplay)
    (bset_redisplay, bset_update_mode_line): New functions.
    (message_dolog): Use bset_redisplay.
    (clear_garbaged_frames): Use fset_redisplay.
    (echo_area_display): Use wset_redisplay.
    (buffer_shared_and_changed): Remove.
    (prepare_menu_bars): Call Vpre_redisplay_function before updating
    frame titles.  Compute the actual set of windows redisplayed.
    Don't update frame titles and menu bars for frames that don't need to
    be redisplayed.
    (propagate_buffer_redisplay): New function.
    (AINC): New macro.
    (redisplay_internal): Use it.  Be more selective in the set of windows
    we redisplay.  Propagate windows_or_buffers_changed to
    update_mode_lines a bit later to simplify the code.
    (mark_window_display_accurate_1): Reset window and buffer's
    `redisplay' flag.
    (redisplay_window): Do nothing if neither the window nor the buffer nor
    the frame needs redisplay.
    * src/window.h (struct window): Add `redisplay' field.
    (wset_redisplay, fset_redisplay, bset_redisplay, bset_update_mode_line)
    (redisplay_other_windows, window_list): New declarations.
    * src/window.c (select_window, Fset_window_start): Use wset_redisplay.
    (window_list): Not static any more.
    (grow_mini_window, shrink_mini_window): Use fset_redisplay.
    * src/minibuf.c (read_minibuf_unwind): Don't redisplay everything.
    * src/insdel.c (prepare_to_modify_buffer_1): Use bset_redisplay.
    * src/frame.c (Fmake_frame_visible): Don't redisplay everything.
    * src/frame.h (struct frame): Add `redisplay' field.
    Move `external_menu_bar' bitfield next to other bit-fields.
    (SET_FRAME_GARBAGED): Use fset_redisplay.
    (SET_FRAME_VISIBLE): Don't garbage the frame;
    Use redisplay_other_windows.
    * src/buffer.h (struct buffer): Add `redisplay' field.
    * src/buffer.c (Fforce_mode_line_update): Pay attention to the `all' flag.
    (modify_overlay): Use bset_redisplay.
    * src/alloc.c (gc_sweep): Don't unmark strings while sweeping symbols.
    
    * lisp/doc-view.el (doc-view-goto-page): Update mode-line.

Re: Redisplay problems?

[Kan-Ru Chen (陳侃如)]

Óscar Fuentes <ofv@wanadoo.es> writes:

> 241fd4407576cb9a417a0f0a546a164cf56c927a is the first bad commit
> commit 241fd4407576cb9a417a0f0a546a164cf56c927a
> Author: Stefan Monnier <monnier@iro.umontreal.ca>
> Date:   Thu Nov 28 17:43:09 2013 -0500
>
>     Refine redisplay optimizations to only redisplay *some* frames/windows
>     rather than all of them.

I can confirm this is the first bad commit. The corresponding bzr commit
is

revno: 115272
committer: Stefan Monnier <monnier@iro.umontreal.ca>
branch nick: trunk
timestamp: Thu 2013-11-28 17:43:09 -0500
message:
  Refine redisplay optimizations to only redisplay *some* frames/windows
  rather than all of them.


Kanru

Re: Redisplay problems?

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 792 bytes --]

 > I find a way to reproduce it reliably. Just run
 >
 > emacs -Q --eval "(progn (scroll-bar-mode -1) \
 >                         (iconify-or-deiconify-frame) \
 >                         (sleep-for 1))"
 >
 > Version (compiled from bzr):
 > GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18
 >
 > then deiconify the opened frame. The cursor is gone. The frame does not
 > redisplay correctly either.

Does the attached patch fix it?  If so, we probably need a better way to avoid

              If we where iconified, we should not set garbaged,
              because that stops redrawing on Expose events.  This looks
              bad if we are called from a recursive event loop
              (x_dispatch_event), for example when a dialog is up.

martin

[-- Attachment #2: xterm.diff --]
[-- Type: text/plain, Size: 520 bytes --]

=== modified file 'src/xterm.c'
--- src/xterm.c	2014-03-11 06:50:01 +0000
+++ src/xterm.c	2014-03-19 16:18:14 +0000
@@ -6148,7 +6148,7 @@
              because that stops redrawing on Expose events.  This looks
              bad if we are called from a recursive event loop
              (x_dispatch_event), for example when a dialog is up.  */
-          if (!iconified)
+/**           if (!iconified) **/
             SET_FRAME_GARBAGED (f);

           /* Check if fullscreen was specified before we where mapped the

Re: Redisplay problems?

[Óscar Fuentes]

martin rudalics <rudalics@gmx.at> writes:

>> I find a way to reproduce it reliably. Just run
>>
>> emacs -Q --eval "(progn (scroll-bar-mode -1) \
>>                         (iconify-or-deiconify-frame) \
>>                         (sleep-for 1))"
>>
>> Version (compiled from bzr):
>> GNU Emacs 24.3.50.7 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10) of 2014-03-18
>>
>> then deiconify the opened frame. The cursor is gone. The frame does not
>> redisplay correctly either.
>
> Does the attached patch fix it?  If so, we probably need a better way to avoid
>
>              If we where iconified, we should not set garbaged,
>              because that stops redrawing on Expose events.  This looks
>              bad if we are called from a recursive event loop
>              (x_dispatch_event), for example when a dialog is up.
>
> martin
> === modified file 'src/xterm.c'
> --- src/xterm.c	2014-03-11 06:50:01 +0000
> +++ src/xterm.c	2014-03-19 16:18:14 +0000
> @@ -6148,7 +6148,7 @@
>               because that stops redrawing on Expose events.  This looks
>               bad if we are called from a recursive event loop
>               (x_dispatch_event), for example when a dialog is up.  */
> -          if (!iconified)
> +/**           if (!iconified) **/
>              SET_FRAME_GARBAGED (f);
>
>            /* Check if fullscreen was specified before we where mapped the

Yes, the test case works correctly with that change.

Re: Redisplay problems?

[Stefan]

>> -          if (!iconified)
>> +/**           if (!iconified) **/
[...]
> Yes, the test case works correctly with that change.

I don't yet understand what's the root cause of the problem, but I get
the impression that the above is a workaround rather than a fix.


        Stefan

Re: Redisplay problems?

[Stefan]

> Does the attached patch fix it?

The patch below seems to fix the problem (at least the test case).
Can someone confirm that it's a good idea?

> If so, we probably need a better way to avoid
>
>              If we where iconified, we should not set garbaged,
>              because that stops redrawing on Expose events.  This looks
>              bad if we are called from a recursive event loop
>              (x_dispatch_event), for example when a dialog is up.

I think my patch indeed fixes this problem by distinguishing between
f->garbaged (which presumably stops redrawing in Expose, tho I don't
offhand see where that happens) and frame_garbaged (which is needed to
get the next redisplay to be properly triggered and performed).

If it looks right, then we might reconsider the SET_FRAME_GARBAGED in
w32term.c in light of such a change, but that's even further away from
my expertise.


        Stefan


--- src/frame.h	2014-01-01 07:43:34 +0000
+++ src/frame.h	2014-03-20 04:30:42 +0000
@@ -946,6 +946,9 @@
       }								\
   } while (false)
 
+/* False means there are no visible garbaged frames.  */
+extern bool frame_garbaged;
+
 /* Set visibility of frame F.
    We call redisplay_other_windows to make sure the frame gets redisplayed
    if some changes were applied to it while it wasn't visible (and hence
@@ -955,8 +958,13 @@
 SET_FRAME_VISIBLE (struct frame *f, int v)
 {
   eassert (0 <= v && v <= 2);
+  if (v)
+    {
   if (v == 1 && f->visible != 1)
     redisplay_other_windows ();
+      if (FRAME_GARBAGED_P (f))
+	frame_garbaged = true;
+    }
   f->visible = v;
 }
 
@@ -972,9 +980,6 @@
 extern Lisp_Object Qterminal;
 extern Lisp_Object Qnoelisp;
 
-/* True means there is at least one garbaged frame.  */
-extern bool frame_garbaged;
-
 extern void set_menu_bar_lines (struct frame *, Lisp_Object, Lisp_Object);
 extern struct frame *decode_window_system_frame (Lisp_Object);
 extern struct frame *decode_live_frame (Lisp_Object);

--- src/xterm.c	2014-03-11 06:50:01 +0000
+++ src/xterm.c	2014-03-20 04:28:16 +0000
@@ -6142,14 +6142,6 @@
       if (f)
         {
 	  bool iconified = FRAME_ICONIFIED_P (f);
-          /* wait_reading_process_output will notice this and update
-             the frame's display structures.
-             If we where iconified, we should not set garbaged,
-             because that stops redrawing on Expose events.  This looks
-             bad if we are called from a recursive event loop
-             (x_dispatch_event), for example when a dialog is up.  */
-          if (!iconified)
-            SET_FRAME_GARBAGED (f);
 
           /* Check if fullscreen was specified before we where mapped the
              first time, i.e. from the command line.  */

Re: Redisplay problems?

[martin rudalics]

 > I don't yet understand what's the root cause of the problem, but I get
 > the impression that the above is a workaround rather than a fix.

Probably.  It just mimics what the w32 build does in this case.
And we'd still have to know whether it solves the OP's problem.


Glenn: Can we mix this with

Bug#16684: Sometimes a frame is not drawn properly

The symptoms are the same and we could get the present thread a
bug number.

martin

Re: Redisplay problems?

[martin rudalics]

 > I think my patch indeed fixes this problem by distinguishing between
 > f->garbaged (which presumably stops redrawing in Expose, tho I don't
 > offhand see where that happens) and frame_garbaged (which is needed to
 > get the next redisplay to be properly triggered and performed).

IIUC my earlier proposal to SET_FRAME_GARBAGED unconditionally subsumes
what you propose here because I set frame_garbaged and the garbaged slot
of the frame that received the MapNotify event.  However, since I set
the garbaged slot unconditionally, your proposal provides a better check
as to whether we previously have set a frame's garbaged slot correctly.
Is that observation correct or am I missing something?

 > If it looks right, then we might reconsider the SET_FRAME_GARBAGED in
 > w32term.c in light of such a change, but that's even further away from
 > my expertise.

I'm still struggling with the semantics of what a "garbaged frame" is
and whether we always set the garbaged slot correctly.

martin

Re: Redisplay problems?

[Stefan]

>> I think my patch indeed fixes this problem by distinguishing between
>> f->garbaged (which presumably stops redrawing in Expose, tho I don't
>> offhand see where that happens) and frame_garbaged (which is needed to
>> get the next redisplay to be properly triggered and performed).
> IIUC my earlier proposal to SET_FRAME_GARBAGED unconditionally subsumes
> what you propose here because I set frame_garbaged and the garbaged slot
> of the frame that received the MapNotify event.  However, since I set
> the garbaged slot unconditionally, your proposal provides a better check
> as to whether we previously have set a frame's garbaged slot correctly.
> Is that observation correct or am I missing something?

Yes, tho I'm not sure I agree with "subsumes": according to the comment,
setting more things "garbaged" may cause some frame's content to stay
blank for a while (because Emacs is in the middle of something which
prevents redisplay from taking place).

And of course, the frame's "garbaged" bit may not always be needed: if
the frame was simply iconified+deiconified without any other change,
there's no need to recompute matrices nor redraw anything, since it's
pretty much the same as obscuring the frame with another and then
exposing it again.

>> If it looks right, then we might reconsider the SET_FRAME_GARBAGED in
>> w32term.c in light of such a change, but that's even further away from
>> my expertise.
> I'm still struggling with the semantics of what a "garbaged frame" is
> and whether we always set the garbaged slot correctly.

I'm not 100% clear either, but my current understanding is that
"garbaged" means that the frame needs to be fully redrawn in the
following sense:

Normal redisplay takes place by first computing new matrices, then
comparing the old matrices to the new matrices to discover what needs to
be changed on screen, then redrawing the parts that have changed.

So "garbaged" means that we should not try to only redraw the parts that
have changed.

Note that the drawing that takes place in response to Expose events is
not a "redraw": "redraw" is when a change inside Emacs causes a need to
change the display, whereas expose events are due to changes outside
of Emacs.

Part of the reason it's still fuzzy is that xdisp.c seems to recompute
the matrices when it finds a "garbaged" frame, so it seems that it
doesn't just cause it to "redraw".  I have the impression that this is
a mistake in that it's more work than needed, and also in that some code
relies on that behavior (i.e. it sets the bit to cause a complete
redisplay+redraw).


        Stefan

Re: Redisplay problems?

[Stefan]

> And of course, the frame's "garbaged" bit may not always be needed: if
> the frame was simply iconified+deiconified without any other change,
> there's no need to recompute matrices nor redraw anything, since it's
> pretty much the same as obscuring the frame with another and then
> exposing it again.

I suggest the patch below for w32term.c.  Guaranteed 100% untested, of
course,


        Stefan


=== modified file 'src/w32term.c'
--- src/w32term.c	2014-03-14 10:38:46 +0000
+++ src/w32term.c	2014-03-20 13:55:14 +0000
@@ -4184,8 +4184,8 @@
 
 	  if (f)
 	    {
-	      if (msg.rect.right == msg.rect.left ||
-		  msg.rect.bottom == msg.rect.top)
+	      if (msg.rect.right == msg.rect.left
+		  || msg.rect.bottom == msg.rect.top)
 		{
 		  /* We may get paint messages even though the client
 		     area is clipped - these are not expose events. */
@@ -4199,7 +4199,6 @@
 		  /* Definitely not obscured, so mark as visible.  */
 		  SET_FRAME_VISIBLE (f, 1);
 		  SET_FRAME_ICONIFIED (f, 0);
-		  SET_FRAME_GARBAGED (f);
 		  if (!f->output_data.w32->asked_for_visible)
 		    DebPrint (("frame %p (%s) reexposed by WM_PAINT\n", f,
 			       SDATA (f->name)));
@@ -4444,8 +4443,9 @@
 #if 0 /* The below is an invalid comparison when CHECK_LISP_OBJECT_TYPE.
 	 But it was originally changed to this to fix a bug, so I have
 	 not removed it completely in case the bug is still there.  */
-          if (help_echo_string != previous_help_echo_string ||
-	      (!NILP (help_echo_string) && !STRINGP (help_echo_string) && f->mouse_moved))
+          if (help_echo_string != previous_help_echo_string
+	      || (!NILP (help_echo_string) && !STRINGP (help_echo_string)
+		  && f->mouse_moved))
 #else /* This is what xterm.c does.  */
 	    if (!NILP (help_echo_string)
 		|| !NILP (previous_help_echo_string))
@@ -4557,8 +4557,8 @@
 
 	case WM_VSCROLL:
 	  {
-	    struct scroll_bar *bar =
-	      x_window_to_scroll_bar ((HWND)msg.msg.lParam);
+	    struct scroll_bar *bar
+	      = x_window_to_scroll_bar ((HWND)msg.msg.lParam);
 
 	    if (bar)
 	      w32_scroll_bar_handle_click (bar, &msg, &inev);
@@ -4645,8 +4645,9 @@
 		    SET_FRAME_VISIBLE (f, 1);
 		    SET_FRAME_ICONIFIED (f, 0);
 
-		    /* wait_reading_process_output will notice this
-		       and update the frame's display structures.  */
+		    /* FIXME: We should not need to set `garbaged' if the
+		       window was simply deiconified and it was previously
+		       maximized.  */
 		    SET_FRAME_GARBAGED (f);
 
 		    if (iconified)
@@ -4692,8 +4693,9 @@
 		      SET_FRAME_VISIBLE (f, 1);
 		    SET_FRAME_ICONIFIED (f, 0);
 
-		    /* wait_reading_process_output will notice this
-		       and update the frame's display structures.  */
+		    /* FIXME: We should not need to set `garbaged' if the
+		       window was simply deiconified and it had previously
+		       the same size.  */
 		    SET_FRAME_GARBAGED (f);
 
 		    if (iconified)
@@ -4990,7 +4992,6 @@
 
 		if (obscured)
 		  {
-		    SET_FRAME_GARBAGED (f);
 		    DebPrint (("obscured frame %p (%s) found to be visible\n",
 			       f, SDATA (f->name)));
 

Re: Redisplay problems?

[Glenn Morris]

martin rudalics wrote:

> Glenn: Can we mix this with
>
> Bug#16684: Sometimes a frame is not drawn properly
>
> The symptoms are the same and we could get the present thread a
> bug number.

If you are asking me to do something, I don't know what you want. :)

BTW, sending "I think maybe possibly perhaps this is a bug" emails to
emacs-devel doesn't save anyone work. Just make a bug-gnu-emacs report
from the start.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Date: Thu, 20 Mar 2014 00:46:13 -0400
> Cc: Christian Lynbech <christian@defun.dk>, James Cloos <cloos@jhcloos.com>,
> 	Kan-Ru Chen (陳侃如) <kanru@kanru.info>,
> 	emacs-devel@gnu.org
> 
> > Does the attached patch fix it?
> 
> The patch below seems to fix the problem (at least the test case).
> Can someone confirm that it's a good idea?
> 
> > If so, we probably need a better way to avoid
> >
> >              If we where iconified, we should not set garbaged,
> >              because that stops redrawing on Expose events.  This looks
> >              bad if we are called from a recursive event loop
> >              (x_dispatch_event), for example when a dialog is up.
> 
> I think my patch indeed fixes this problem by distinguishing between
> f->garbaged (which presumably stops redrawing in Expose, tho I don't
> offhand see where that happens) and frame_garbaged (which is needed to
> get the next redisplay to be properly triggered and performed).

If you are worried about doing too much when a frame is deiconified,
why do you want a hammer such as frame_garbaged?  This sounds like a
contradiction to me.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Date: Thu, 20 Mar 2014 10:01:17 -0400
> Cc: Christian Lynbech <christian@defun.dk>, James Cloos <cloos@jhcloos.com>,
> 	Kan-Ru Chen (陳侃如) <kanru@kanru.info>,
> 	emacs-devel@gnu.org
> 
> > And of course, the frame's "garbaged" bit may not always be needed: if
> > the frame was simply iconified+deiconified without any other change,
> > there's no need to recompute matrices nor redraw anything, since it's
> > pretty much the same as obscuring the frame with another and then
> > exposing it again.
> 
> I suggest the patch below for w32term.c.  Guaranteed 100% untested, of
> course,

Thanks.  What bug does this fix?

Re: Redisplay problems?

[Stefan]

>>> > emacs -Q --eval "(progn (scroll-bar-mode -1) \
>>> >                         (iconify-or-deiconify-frame) \
>>> >                         (sleep-for 1))"

I installed the fix I posted earlier.
Please make a proper bug report if you still see such problems,


        Stefan

Re: Redisplay problems?

[Stefan]

>> > And of course, the frame's "garbaged" bit may not always be needed: if
>> > the frame was simply iconified+deiconified without any other change,
>> > there's no need to recompute matrices nor redraw anything, since it's
>> > pretty much the same as obscuring the frame with another and then
>> > exposing it again.
>> I suggest the patch below for w32term.c.  Guaranteed 100% untested, of
>> course,
> Thanks.  What bug does this fix?

Nothing, IIUC.  It just "tightens" the code to try and clarify when
"garbaged" needs to be set.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Date: Thu, 20 Mar 2014 08:45:59 -0400
> Cc: Christian Lynbech <christian@defun.dk>, James Cloos <cloos@jhcloos.com>,
> 	Kan-Ru Chen (陳侃如) <kanru@kanru.info>,
> 	emacs-devel@gnu.org
> 
> And of course, the frame's "garbaged" bit may not always be needed: if
> the frame was simply iconified+deiconified without any other change,
> there's no need to recompute matrices nor redraw anything, since it's
> pretty much the same as obscuring the frame with another and then
> exposing it again.

But exposing does require a kind of "redrawing", see below.

> I'm not 100% clear either, but my current understanding is that
> "garbaged" means that the frame needs to be fully redrawn in the
> following sense:
> 
> Normal redisplay takes place by first computing new matrices, then
> comparing the old matrices to the new matrices to discover what needs to
> be changed on screen, then redrawing the parts that have changed.

Your description is slightly inaccurate, and on top of that, you use
several keywords in confusingly overloaded senses, which makes this
discussion harder to understand.

So allow me first to set the stage by making the description more
accurate.

Redisplay includes 2 phases.  In the first phase, Emacs computes, for
every window, the desired glyph matrix, which describes what should be
on the display.  This stage includes some redisplay optimizations,
which can decide that certain portions of the display couldn't
possibly have changed, in which case the corresponding parts ("glyph
rows") of the desired matrix are marked "disabled".

In the second phase of redisplay, Emacs compares the desired matrix
with the "current matrix" (which was the desired matrix during the
previous redisplay cycle).  This stage also includes optimizations,
but of a different kind.  For each screen line, which corresponds to a
glyph row, Emacs redraws only the parts of that glyph row that are
different from the corresponding row of the current matrix, and it
also reuses what's on display as much as possible, even if they don't
match exactly.

Crucially, if the current matrix' glyph row is "disabled", the
corresponding screen line is redrawn in its entirety.  (If the desired
matrix' glyph row is "disabled", it is simply copied to the current
matrix without redrawing the line.)

When Emacs decides that some part of a glyph row needs to be redrawn,
it calls a function named draw_glyphs, which actually delivers the
glyphs to the glass by calling the display-specific back-end (xterm.c
etc.), after arranging the row's glyphs in "glyph strings", which make
the job easier for the back end.

> So "garbaged" means that we should not try to only redraw the parts that
> have changed.

When redisplay finds that a frame is "garbaged", it marks all of its
glyph matrix rows "disabled".  This will force their complete
redrawing, as described above.

> Note that the drawing that takes place in response to Expose events is
> not a "redraw": "redraw" is when a change inside Emacs causes a need to
> change the display, whereas expose events are due to changes outside
> of Emacs.

An expose event calls expose_frame, which walks the frame's window
tree, and redraws the part of each window that belongs to the exposed
rectangle.  This doesn't compute the glyph matrices at all, but just
calls draw_glyphs directly to redraw the portions of the glyph rows
that were exposed.  IOW, the existing current glyph matrix is used
without recomputing it.

> Part of the reason it's still fuzzy is that xdisp.c seems to recompute
> the matrices when it finds a "garbaged" frame, so it seems that it
> doesn't just cause it to "redraw".  I have the impression that this is
> a mistake in that it's more work than needed, and also in that some code
> relies on that behavior (i.e. it sets the bit to cause a complete
> redisplay+redraw).

I don't think I understand what you meant to say here.  But if you
were talking about what to do about a frame that was deiconified, I
think you need to call expose_frame, and arrange for the exposed
rectangle to cover the entire frame.  That should do what you want, no
more, no less.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  cloos@jhcloos.com,  kanru@kanru.info,  emacs-devel@gnu.org
> Date: Thu, 20 Mar 2014 14:00:47 -0400
> 
> >> > And of course, the frame's "garbaged" bit may not always be needed: if
> >> > the frame was simply iconified+deiconified without any other change,
> >> > there's no need to recompute matrices nor redraw anything, since it's
> >> > pretty much the same as obscuring the frame with another and then
> >> > exposing it again.
> >> I suggest the patch below for w32term.c.  Guaranteed 100% untested, of
> >> course,
> > Thanks.  What bug does this fix?
> 
> Nothing, IIUC.  It just "tightens" the code to try and clarify when
> "garbaged" needs to be set.

Please tell if you still think it is needed, after you read my
explanations about "redisplay", "redraw", "garbaged", and "expose".

Re: Redisplay problems?

[martin rudalics]

 > Yes, tho I'm not sure I agree with "subsumes": according to the comment,
 > setting more things "garbaged" may cause some frame's content to stay
 > blank for a while (because Emacs is in the middle of something which
 > prevents redisplay from taking place).

OK.  But the major issue is to prevent a frame's contents to stay blank
forever (unless the user intervenes in some way or the other).

 > And of course, the frame's "garbaged" bit may not always be needed: if
 > the frame was simply iconified+deiconified without any other change,
 > there's no need to recompute matrices nor redraw anything, since it's
 > pretty much the same as obscuring the frame with another and then
 > exposing it again.

Yes, that's the main difference.  But it requires to carefully keep
track of changes via the `garbaged' slot.

 > I'm not 100% clear either, but my current understanding is that
 > "garbaged" means that the frame needs to be fully redrawn

... where "fully" means all windows?

 > in the
 > following sense:
 >
 > Normal redisplay takes place by first computing new matrices, then
 > comparing the old matrices to the new matrices to discover what needs to
 > be changed on screen, then redrawing the parts that have changed.
 >
 > So "garbaged" means that we should not try to only redraw the parts that
                                      ^^^

 > have changed.

I'm not sure I follow you here.  If we do a resize of the frame we may
need larger matrices so we tell the display engine via the resized_p
slot.  If a change is restricted to a certain buffer or window only, we
can tell the display engien which one is affected.  Other from that what
kind of guidance can you possibly give to the display engine?

 > Note that the drawing that takes place in response to Expose events is
 > not a "redraw": "redraw" is when a change inside Emacs causes a need to
 > change the display, whereas expose events are due to changes outside
 > of Emacs.

But it's a redraw when we expose a hitherto invisible/obscured frame
whose contents have changed while it was invisible/obscured.

 > Part of the reason it's still fuzzy is that xdisp.c seems to recompute
 > the matrices when it finds a "garbaged" frame,

But we do have to compute the new matrices to know whether they have
changed.  I'm fully confused now.

 > so it seems that it
 > doesn't just cause it to "redraw".  I have the impression that this is
 > a mistake in that it's more work than needed, and also in that some code
 > relies on that behavior (i.e. it sets the bit to cause a complete
 > redisplay+redraw).

What is the "less work" that's needed?

martin

Re: Redisplay problems?

[martin rudalics]

 >> Glenn: Can we mix this with
 >>
 >> Bug#16684: Sometimes a frame is not drawn properly
 >>
 >> The symptoms are the same and we could get the present thread a
 >> bug number.
 >
 > If you are asking me to do something, I don't know what you want. :)

Just the impossible - rewrite history.

 > BTW, sending "I think maybe possibly perhaps this is a bug" emails to
 > emacs-devel doesn't save anyone work. Just make a bug-gnu-emacs report
 > from the start.

I know.  But here we have a bug report, namely that of bug#16684 which
describes the same symptoms.  And people preferred to react only to the
present thread.

martin

Re: Redisplay problems?

[Óscar Fuentes]

Stefan <monnier@iro.umontreal.ca> writes:

>>>> > emacs -Q --eval "(progn (scroll-bar-mode -1) \
>>>> >                         (iconify-or-deiconify-frame) \
>>>> >                         (sleep-for 1))"
>
> I installed the fix I posted earlier.
> Please make a proper bug report if you still see such problems,

You sent that message to me but please note that I'm not neither the
original reporter nor the author of the test case. Moreover, I was not
affected by the bug because my config does not trigger the problem.

I cc'ed both users to make sure they receive your suggestion message.

Re: Redisplay problems?

[Eli Zaretskii]

> Date: Thu, 20 Mar 2014 20:22:38 +0100
> From: martin rudalics <rudalics@gmx.at>
> Cc: Christian Lynbech <christian@defun.dk>, James Cloos <cloos@jhcloos.com>,
> 	"Kan-Ru Chen (陳侃如)" <kanru@kanru.info>,
> 	emacs-devel@gnu.org
> 
>  > And of course, the frame's "garbaged" bit may not always be needed: if
>  > the frame was simply iconified+deiconified without any other change,
>  > there's no need to recompute matrices nor redraw anything, since it's
>  > pretty much the same as obscuring the frame with another and then
>  > exposing it again.
> 
> Yes, that's the main difference.  But it requires to carefully keep
> track of changes via the `garbaged' slot.

As I wrote elsewhere, I see no reason for marking such a frame
"garbaged", because its glyph matrices are just fine.  Unless you can
show me some code that avoids recomputing or invalidating the matrices
of an iconified frame.

>  > I'm not 100% clear either, but my current understanding is that
>  > "garbaged" means that the frame needs to be fully redrawn
> 
> ... where "fully" means all windows?

Yes, all of them.  The glyph matrices of all the windows of such a
frame are invalidated by clear_garbaged_frames, which is called at the
beginning of a redisplay cycle.

>  > Normal redisplay takes place by first computing new matrices, then
>  > comparing the old matrices to the new matrices to discover what needs to
>  > be changed on screen, then redrawing the parts that have changed.
>  >
>  > So "garbaged" means that we should not try to only redraw the parts that
>                                       ^^^
> 
>  > have changed.
> 
> I'm not sure I follow you here.

I think Stefan wanted to say that "garbaged" means redraw more than
just the parts that have changed.

If you read my description, I explained that a "garbaged" frame means
Emacs does not believe the contents of the current glyph matrices for
that frame is reliable.  So it discards them, and that forces a
complete redraw of all the windows on the frame.

>  > Note that the drawing that takes place in response to Expose events is
>  > not a "redraw": "redraw" is when a change inside Emacs causes a need to
>  > change the display, whereas expose events are due to changes outside
>  > of Emacs.
> 
> But it's a redraw when we expose a hitherto invisible/obscured frame
> whose contents have changed while it was invisible/obscured.

If the glyph matrices of such a frame were updated when the contents
changed, then there's no need to recompute them at expose event time.

>  > Part of the reason it's still fuzzy is that xdisp.c seems to recompute
>  > the matrices when it finds a "garbaged" frame,
> 
> But we do have to compute the new matrices to know whether they have
> changed.  I'm fully confused now.

Actually, the desired glyph matrices are always recomputed when we
enter redisplay.  But if the result is identical to the current
matrix, then nothing is redrawn.

When a frame is marked "garbaged", Emacs clears the current glyph
matrices, so it loses the ability to optimize the redraw phase of the
redisplay cycle, because there's nothing with which to compare the
desired matrices.

Re: Redisplay problems?

[Stefan Monnier]

>> So "garbaged" means that we should not try to only redraw the parts that
>> have changed.
> When redisplay finds that a frame is "garbaged", it marks all of its
> glyph matrix rows "disabled".  This will force their complete
> redrawing, as described above.

Right, but that's not how "garbaged" is described (e.g. in the comment
before it: "True if this frame should be redrawn"), hence the fuzzyness.

The way I see it described, "garbaged" means that the display is not in
sync with the matrices any more, but not that the matrices need to
be recomputed.

But the implementation you describe here implies that "garbaged" will
end up recomputing all the matrices.

And in turn I suspect some of the code takes advantage of the knowledge
of how "garbaged" is implemented: it sets "garbaged" to cause the
matrices to be recomputed (and the frame redrawn redrawn).

> I don't think I understand what you meant to say here.  But if you
> were talking about what to do about a frame that was deiconified, I
> think you need to call expose_frame, and arrange for the exposed
> rectangle to cover the entire frame.  That should do what you want, no
> more, no less.

Doesn't the windowing-system do that already for us?


        Stefan

Re: Redisplay problems?

[Stefan Monnier]

> But it's a redraw when we expose a hitherto invisible/obscured frame
> whose contents have changed while it was invisible/obscured.

Yes, of course, but you can look at it as "deiconify with the old
display content" (without recomputing matrices nor redrawing) followed
by a normal redisplay (which may recompute matrices and/or redraw
depending on whether something actually did change while the frame was
invisible).

>> Part of the reason it's still fuzzy is that xdisp.c seems to recompute
>> the matrices when it finds a "garbaged" frame,
> But we do have to compute the new matrices to know whether they have
> changed.  I'm fully confused now.

No, if the "redisplay" bits are "false" we know that nothing has changed.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 20 Mar 2014 16:55:52 -0400
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
> 	emacs-devel@gnu.org
> 
> >> So "garbaged" means that we should not try to only redraw the parts that
> >> have changed.
> > When redisplay finds that a frame is "garbaged", it marks all of its
> > glyph matrix rows "disabled".  This will force their complete
> > redrawing, as described above.
> 
> Right, but that's not how "garbaged" is described (e.g. in the comment
> before it: "True if this frame should be redrawn"), hence the fuzzyness.

I never said your fuzzyness is not justified.  That's why I described
how things really work.

> The way I see it described, "garbaged" means that the display is not in
> sync with the matrices any more, but not that the matrices need to
> be recomputed.
> 
> But the implementation you describe here implies that "garbaged" will
> end up recomputing all the matrices.

When the display is messed up (for some value of "messed up"), we
cannot trust the current glyph matrices anymore.  Another situation is
frame resize: then we _know_ the current matrices are useless.  In
these cases, we must redisplay everything, which must start from
computing the desired glyph matrices.  IOW, Emacs has no way of
performing redisplay, except by computing the desired matrices --
these tell what should be on the screen.

The commentary describes the effect of the "garbaged" flag
(eventually, the frame will indeed be redrawn in its entirety), but it
does not describe the implementation of causing that effect.

> And in turn I suspect some of the code takes advantage of the knowledge
> of how "garbaged" is implemented: it sets "garbaged" to cause the
> matrices to be recomputed (and the frame redrawn redrawn).

As I described, the desired matrices are _always_ recomputed, if we
enter redisplay through redisplay_internal.  (Expose events bypass
that stage.)  So there's no need to cause the matrices to be
recomputed: they always will be.  What you can do is disable the
optimizations that might allow Emacs to recompute only some portions
of the desired matrices.  We currently do that by setting
windows_or_buffers_changed (via fset_redisplay that calls
redisplay_other_windows), when the frame's "garbaged" flag is set.
This disabling of redisplay optimizations is inevitable when the
current matrices cannot be used for comparison with desired ones.  But
if you only want to cause a full re-computation of all the desired
matrices, all you need to do is set windows_or_buffers_changed.  Note,
however, that doing that without also setting the frame's "garbaged"
flag might end up redrawing very little, or nothing at all, if the
recomputed desired matrices are found to be similar or identical to
the current one.  Only setting the "garbaged" flag ensures we will
actually redraw the entire frame, i.e. refresh what's on the glass.

> > I don't think I understand what you meant to say here.  But if you
> > were talking about what to do about a frame that was deiconified, I
> > think you need to call expose_frame, and arrange for the exposed
> > rectangle to cover the entire frame.  That should do what you want, no
> > more, no less.
> 
> Doesn't the windowing-system do that already for us?

If you mean that it somehow magically calls expose_frame, then no, I
don't think so.  We should call it ourselves when the right event(s)
come(s).  I'm not an expert on those parts of Emacs, but it looks to
me that we don't consistently call expose_frame when a frame is
deiconified.  I might be mistaken, though.

Also note that setting the "garbaged" flag makes expose_frame a no-op:
it immediately returns without doing anything, because it knows the
frame will be completely redrawn very soon.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 20 Mar 2014 17:00:49 -0400
> Cc: Christian Lynbech <christian@defun.dk>, James Cloos <cloos@jhcloos.com>,
> 	Kan-Ru Chen (陳侃如) <kanru@kanru.info>,
> 	emacs-devel@gnu.org
> 
> > But it's a redraw when we expose a hitherto invisible/obscured frame
> > whose contents have changed while it was invisible/obscured.
> 
> Yes, of course, but you can look at it as "deiconify with the old
> display content" (without recomputing matrices nor redrawing) followed
> by a normal redisplay (which may recompute matrices and/or redraw
> depending on whether something actually did change while the frame was
> invisible).

That would cause an unpleasant momentary display of wrong contents.

Anyway, I very much doubt that an iconified frame might have its
current matrices outdated.  What is the use case for this to happen?

> >> Part of the reason it's still fuzzy is that xdisp.c seems to recompute
> >> the matrices when it finds a "garbaged" frame,
> > But we do have to compute the new matrices to know whether they have
> > changed.  I'm fully confused now.
> 
> No, if the "redisplay" bits are "false" we know that nothing has changed.

Are these bits reset for windows on iconified frames?  If so, you have
no other way but marking such frames "garbaged" when they are deiconified.

Re: Redisplay problems?

[martin rudalics]

 >> But it's a redraw when we expose a hitherto invisible/obscured frame
 >> whose contents have changed while it was invisible/obscured.
 >
 > If the glyph matrices of such a frame were updated when the contents
 > changed, then there's no need to recompute them at expose event time.

But wasn't the whole idea of maintaining an "obscured" state to not
update the glyph matrices when a change occurs while the frame is
obscured but instead to wait until the frame gets exposed again?

martin

Re: Redisplay problems?

[Eli Zaretskii]

> Date: Fri, 21 Mar 2014 09:03:39 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: kanru@kanru.info, christian@defun.dk, monnier@iro.umontreal.ca, 
>  cloos@jhcloos.com, emacs-devel@gnu.org
> 
>  >> But it's a redraw when we expose a hitherto invisible/obscured frame
>  >> whose contents have changed while it was invisible/obscured.
>  >
>  > If the glyph matrices of such a frame were updated when the contents
>  > changed, then there's no need to recompute them at expose event time.
> 
> But wasn't the whole idea of maintaining an "obscured" state to not
> update the glyph matrices when a change occurs while the frame is
> obscured but instead to wait until the frame gets exposed again?

Maybe so, but then what exactly is the problem we are discussing here?
If the current glyph matrices are not up to date, marking the frame
"garbaged" is the only way to redisplay it.

Re: Redisplay problems?

[martin rudalics]

 >> But wasn't the whole idea of maintaining an "obscured" state to not
 >> update the glyph matrices when a change occurs while the frame is
 >> obscured but instead to wait until the frame gets exposed again?
 >
 > Maybe so, but then what exactly is the problem we are discussing here?
 > If the current glyph matrices are not up to date, marking the frame
 > "garbaged" is the only way to redisplay it.

Pretty early in this thread you stated

    Not reproducible here, but I'm not on GNU/Linux with GTK+.

and I proposed a fix to do on GNU/Linux what we do on Windows but Stefan
considered that too heavy because it sets a frame garbaged even when it
was only iconified for some short period and its contentes didn't change
at all.  So Stefan committed another fix for GNU/Linux and now proposes
to backport a similar fix to Windows.  And I can't tell much because the
semantics of "garbaged" are still eluding me.

martin

Re: Redisplay problems?

[Eli Zaretskii]

> Date: Fri, 21 Mar 2014 10:51:22 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: emacs-devel@gnu.org, christian@defun.dk, monnier@iro.umontreal.ca, 
>  kanru@kanru.info, cloos@jhcloos.com
> 
>  >> But wasn't the whole idea of maintaining an "obscured" state to not
>  >> update the glyph matrices when a change occurs while the frame is
>  >> obscured but instead to wait until the frame gets exposed again?
>  >
>  > Maybe so, but then what exactly is the problem we are discussing here?
>  > If the current glyph matrices are not up to date, marking the frame
>  > "garbaged" is the only way to redisplay it.
> 
> Pretty early in this thread you stated
> 
>     Not reproducible here, but I'm not on GNU/Linux with GTK+.
> 
> and I proposed a fix to do on GNU/Linux what we do on Windows but Stefan
> considered that too heavy because it sets a frame garbaged even when it
> was only iconified for some short period and its contentes didn't change
> at all.  So Stefan committed another fix for GNU/Linux and now proposes
> to backport a similar fix to Windows.  And I can't tell much because the
> semantics of "garbaged" are still eluding me.

I think the solution to Stefan's concerns is to set some flag, perhaps
the frame's redisplay flag, when we bump into an iconified frame
inside redisplay_internal.  And in fact we already do that:

  /* Build desired matrices, and update the display.  If
     consider_all_windows_p is non-zero, do it for all windows on all
     frames.  Otherwise do it for selected_window, only.  */

  if (consider_all_windows_p)
    {
      FOR_EACH_FRAME (tail, frame)
	XFRAME (frame)->updated_p = 0;

      propagate_buffer_redisplay ();

      FOR_EACH_FRAME (tail, frame)
	{
	  struct frame *f = XFRAME (frame);
   ...
	  if (FRAME_WINDOW_P (f) || FRAME_TERMCAP_P (f) || f == sf)
	    {
   ...
	      if (FRAME_VISIBLE_P (f) && !FRAME_OBSCURED_P (f))
		redisplay_windows (FRAME_ROOT_WINDOW (f));
	      /* Remember that the invisible frames need to be redisplayed next
		 time they're visible.  */
	      else if (!REDISPLAY_SOME_P ())
		f->redisplay = true; <<<<<<<<<<<<<<<<<<<<<<<<<<<

So it is unclear to me why there are problems with such frames: their
redisplay flag should already be set when the contents changed, and
reset otherwise.

So why do we need to change anything at all?  A new frame starts with
its garbaged flag set, so it should have been redrawn completely when
it becomes visible the first time.  If that flag is reset, the culprit
might be in the code which resets that flag.  Or maybe
consider_all_windows_p is zero in this case.

I guess some more debugging is in order, because I don't understand
the problem.

Also, I wonder if there are any real-life use cases that trigger this
problem, because the original recipe sounds artificial to me.

Re: Redisplay problems?

[Stefan]

> When the display is messed up (for some value of "messed up"), we
> cannot trust the current glyph matrices anymore.

It all depends on the reason for the "messed up".  E.g. it can be messed
up because some code drew something on screen without going through
"change matrices and reflect that change on screen".  That's what
I would understand as the "natural meaning" for "garbaged" (and in that
case the matrices might still be trusted to reflect "what we'd like to
see on screen").

So, IIUC this "natural meaning" is wrong.  And "garbaged" is really
meant to say "disable all optimizations".
Which then begs the question: what is the difference with
`prevent_redisplay_optimizations_p'?

>> > think you need to call expose_frame, and arrange for the exposed
>> > rectangle to cover the entire frame.  That should do what you want, no
>> > more, no less.
>> Doesn't the windowing-system do that already for us?
> If you mean that it somehow magically calls expose_frame, then no, I
> don't think so.

No, I mean that the windowing-system should already arrange to send us
the relevant expose events (which could be "no events at all", if the
windowing system were to draw the frame from some backed up copy).

> I'm not an expert on those parts of Emacs, but it looks to me that we
> don't consistently call expose_frame when a frame is deiconified.

Indeed and I think that's right because the expose_frame calls will be
performed later in response to expose events.


        Stefan

Re: Redisplay problems?

[Stefan]

>> No, if the "redisplay" bits are "false" we know that nothing has changed.
> Are these bits reset for windows on iconified frames?

Not sure what you mean by "reset", but no iconify&deiconify does not
affect the `redisplay' bits.  And it shouldn't, just like
obscuring&deobscuring a frame doesn't.

> If so, you have no other way but marking such frames "garbaged" when
> they are deiconified.

I highly doubt it.


        Stefan

Re: Redisplay problems?

[Stefan]

> If the current glyph matrices are not up to date, marking the frame
> "garbaged" is the only way to redisplay it.

No: the next redisplay will update the glyph matrices just fine without
having to set `garbaged'.  And the portions of the frame made visible by
unobscuring/deiconifying will be redrawn in response to the expose
events that windowing-system will send us.

> I guess some more debugging is in order, because I don't understand
> the problem.

AFAIK there is no problem (other than people trying to understand
why/how things work).


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  kanru@kanru.info,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Fri, 21 Mar 2014 08:59:37 -0400
> 
> > When the display is messed up (for some value of "messed up"), we
> > cannot trust the current glyph matrices anymore.
> 
> It all depends on the reason for the "messed up".  E.g. it can be messed
> up because some code drew something on screen without going through
> "change matrices and reflect that change on screen".

Emacs doesn't (and cannot) do that, AFAIK.  But maybe I misunderstand
what you mean.

> That's what I would understand as the "natural meaning" for
> "garbaged" (and in that case the matrices might still be trusted to
> reflect "what we'd like to see on screen").

I didn't say it was natural, I tried to explain what it does.

> So, IIUC this "natural meaning" is wrong.  And "garbaged" is really
> meant to say "disable all optimizations".

If you include "smart" redrawing in optimizations, then yes, you could
say that.

> Which then begs the question: what is the difference with
> `prevent_redisplay_optimizations_p'?

That one only prevents optimizations in computing the desired
matrices, AFAIR.  It has no effect on actual redrawing in
update_frame.

> > I'm not an expert on those parts of Emacs, but it looks to me that we
> > don't consistently call expose_frame when a frame is deiconified.
> 
> Indeed and I think that's right because the expose_frame calls will be
> performed later in response to expose events.

Where do you see expose_frame called in response to expose events?
That's what I didn't see.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  cloos@jhcloos.com,  kanru@kanru.info,  emacs-devel@gnu.org
> Date: Fri, 21 Mar 2014 09:02:59 -0400
> 
> > If so, you have no other way but marking such frames "garbaged" when
> > they are deiconified.
> 
> I highly doubt it.

How else can you redisplay a frame whose current glyph matrices fell
behind, while the frame was iconified?

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: martin rudalics <rudalics@gmx.at>,  kanru@kanru.info,  christian@defun.dk,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Fri, 21 Mar 2014 09:08:02 -0400
> 
> > If the current glyph matrices are not up to date, marking the frame
> > "garbaged" is the only way to redisplay it.
> 
> No: the next redisplay will update the glyph matrices just fine without
> having to set `garbaged'.

This is true (except that only parts of the matrices might be
recomputed, which might not be what you want), but computing the
desired glyph matrices when the current matrices cannot be used for
comparison is simply useless, because Emacs will be unable to redraw
the modified parts correctly.  For redrawing, Emacs must either (1)
have current matrices it can trust, or (2) assume the current matrices
are useful, and use the desired matrices to redraw everything.  The
latter alternative is what the garbaged flag triggers.

> And the portions of the frame made visible by
> unobscuring/deiconifying will be redrawn in response to the expose
> events that windowing-system will send us.

That is only possible if the current matrices of the frame correctly
reflect what should be on the screen, because expose_frame uses the
current matrices without recomputing them.

> > I guess some more debugging is in order, because I don't understand
> > the problem.
> 
> AFAIK there is no problem (other than people trying to understand
> why/how things work).

If we don't understand how things work, how can we be sure we fixed
the bug?

Re: Redisplay problems?

[Stefan Monnier]

> Where do you see expose_frame called in response to expose events?
> That's what I didn't see.

xterm.c:

    case Expose:
      f = x_window_to_frame (dpyinfo, event->xexpose.window);
      if (f)
        {
          if (!FRAME_VISIBLE_P (f))
            { ... }
          else
	    {
   #ifdef USE_GTK ... #endif
	      expose_frame (f, event->xexpose.x, event->xexpose.y,
			    event->xexpose.width, event->xexpose.height);

-- Stefan

Re: Redisplay problems?

[Stefan Monnier]

>> > If so, you have no other way but marking such frames "garbaged" when
>> > they are deiconified.
>> I highly doubt it.
> How else can you redisplay a frame whose current glyph matrices fell
> behind, while the frame was iconified?

While the frame is iconified, glyph matrices are not updated, so if some
part of the frame is modified during this time, its `redisplay' bit will
be set (and stay set), so after it gets deiconified, redisplay will know
that the glyph matrices need to be updated.


        Stefan

Re: Redisplay problems?

[Stefan Monnier]

>> And the portions of the frame made visible by
>> unobscuring/deiconifying will be redrawn in response to the expose
>> events that windowing-system will send us.
> That is only possible if the current matrices of the frame correctly
> reflect what should be on the screen,

Why wouldn't it?


        Stefan

Re: Redisplay problems?

[Kan-Ru Chen (陳侃如)]

Eli Zaretskii <eliz@gnu.org> writes:

> Also, I wonder if there are any real-life use cases that trigger this
> problem, because the original recipe sounds artificial to me.

It happens when I restart emacs. Emacs init takes time so I usually
switch to another work-space, for example, to browse
something. Switching work-spaces unmaps(iconify) the emacs window. When
I switch back I get a unusable window unless I resize or restart emacs.

Other people have also encountered this problem so I guess they have
their use cases too.

Kanru

Re: Redisplay problems?

[Stefan Monnier]

>> Also, I wonder if there are any real-life use cases that trigger this
>> problem, because the original recipe sounds artificial to me.
> It happens when I restart emacs. Emacs init takes time so I usually
> switch to another work-space, for example, to browse
> something. Switching work-spaces unmaps(iconify) the emacs window. When
> I switch back I get a unusable window unless I resize or restart emacs.

Can you still reproduce the problem, or is it fixed now?


        Stefan

Re: Redisplay problems?

[Kan-Ru Chen (陳侃如)]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> Also, I wonder if there are any real-life use cases that trigger this
>>> problem, because the original recipe sounds artificial to me.
>> It happens when I restart emacs. Emacs init takes time so I usually
>> switch to another work-space, for example, to browse
>> something. Switching work-spaces unmaps(iconify) the emacs window. When
>> I switch back I get a unusable window unless I resize or restart emacs.
>
> Can you still reproduce the problem, or is it fixed now?

It is fixed for the trunk.

Kanru

Re: Redisplay problems?

[Eli Zaretskii]

> From: Kan-Ru Chen (陳侃如) <kanru@kanru.info>
> Cc: martin rudalics <rudalics@gmx.at>,  emacs-devel@gnu.org,  christian@defun.dk,  monnier@iro.umontreal.ca,  cloos@jhcloos.com
> Date: Sat, 22 Mar 2014 10:00:13 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Also, I wonder if there are any real-life use cases that trigger this
> > problem, because the original recipe sounds artificial to me.
> 
> It happens when I restart emacs. Emacs init takes time so I usually
> switch to another work-space, for example, to browse
> something. Switching work-spaces unmaps(iconify) the emacs window. When
> I switch back I get a unusable window unless I resize or restart emacs.

So you are saying that the use case is having the Emacs startup run
while the initial frame is iconified, is that right?

Thanks.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  kanru@kanru.info,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Fri, 21 Mar 2014 15:31:23 -0400
> 
> > Where do you see expose_frame called in response to expose events?
> > That's what I didn't see.
> 
> xterm.c:
> 
>     case Expose:
>       f = x_window_to_frame (dpyinfo, event->xexpose.window);
>       if (f)
>         {
>           if (!FRAME_VISIBLE_P (f))
>             { ... }
>           else
> 	    {
>    #ifdef USE_GTK ... #endif
> 	      expose_frame (f, event->xexpose.x, event->xexpose.y,
> 			    event->xexpose.width, event->xexpose.height);

This is for visible frames, while we were discussing frames that are
iconified.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  cloos@jhcloos.com,  kanru@kanru.info,  emacs-devel@gnu.org
> Date: Fri, 21 Mar 2014 15:39:17 -0400
> 
> >> > If so, you have no other way but marking such frames "garbaged" when
> >> > they are deiconified.
> >> I highly doubt it.
> > How else can you redisplay a frame whose current glyph matrices fell
> > behind, while the frame was iconified?
> 
> While the frame is iconified, glyph matrices are not updated, so if some
> part of the frame is modified during this time, its `redisplay' bit will
> be set (and stay set), so after it gets deiconified, redisplay will know
> that the glyph matrices need to be updated.

Once again, redisplaying a set of windows needs (a) to compute their
desired matrices, and (b) compare them against the current matrices,
and deliver the differences to the glass.  The 'redisplay' flag takes
care of (a), but if there are no current matrices we can trust,
there's no way to perform (b).  Setting the frame's garbaged flag
tells Emacs that the current matrices are unreliable, and then it
bypasses the matrix comparison in (b), and simply redraws every screen
line in every window of the frame.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  kanru@kanru.info,  christian@defun.dk,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Fri, 21 Mar 2014 15:42:54 -0400
> 
> >> And the portions of the frame made visible by
> >> unobscuring/deiconifying will be redrawn in response to the expose
> >> events that windowing-system will send us.
> > That is only possible if the current matrices of the frame correctly
> > reflect what should be on the screen,
> 
> Why wouldn't it?

Because we stop updating the matrices of iconified and invisible
frames.

Re: Redisplay problems?

[Stefan]

> This is for visible frames, while we were discussing frames that are
> iconified.

AFAIK iconified frames don't get expose events.


        Stefan

Re: Redisplay problems?

[Stefan]

> Once again, redisplaying a set of windows needs (a) to compute their
> desired matrices, and (b) compare them against the current matrices,
> and deliver the differences to the glass.  The 'redisplay' flag takes
> care of (a), but if there are no current matrices we can trust,
> there's no way to perform (b).

But luckily, we can trust the current matrices.


        Stefan

Re: Redisplay problems?

[martin rudalics]

> AFAIK iconified frames don't get expose events.

At least on Windows we have to cater for obscured frames.

martin

Re: Redisplay problems?

[Stefan]

>> > That is only possible if the current matrices of the frame correctly
>> > reflect what should be on the screen,
>> Why wouldn't it?
> Because we stop updating the matrices of iconified and invisible frames.

That does not affect the validity of current matrices, since that only
depends on what the screen contains, and iconify/deiconify does not
affect the screen contents (or rather, if/when/where it does, the
window-system will fix it by sending us corresponding expose events, so
we can still trust the matrices).


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  kanru@kanru.info,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Sat, 22 Mar 2014 09:48:26 -0400
> 
> > This is for visible frames, while we were discussing frames that are
> > iconified.
> 
> AFAIK iconified frames don't get expose events.

Do frames get them when they are deiconified?

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  cloos@jhcloos.com,  kanru@kanru.info,  emacs-devel@gnu.org
> Date: Sat, 22 Mar 2014 09:49:56 -0400
> 
> > Once again, redisplaying a set of windows needs (a) to compute their
> > desired matrices, and (b) compare them against the current matrices,
> > and deliver the differences to the glass.  The 'redisplay' flag takes
> > care of (a), but if there are no current matrices we can trust,
> > there's no way to perform (b).
> 
> But luckily, we can trust the current matrices.

Sometimes we can, sometimes we can't.

(I no longer understand what situation are we talking about.)

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  kanru@kanru.info,  christian@defun.dk,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Sat, 22 Mar 2014 09:56:58 -0400
> 
> >> > That is only possible if the current matrices of the frame correctly
> >> > reflect what should be on the screen,
> >> Why wouldn't it?
> > Because we stop updating the matrices of iconified and invisible frames.
> 
> That does not affect the validity of current matrices

Sometimes it doesn't, sometimes it does.  Look inside redisplay_window
at all the calls to clear_glyph_matrix, for example: all these will be
skipped while the frame is iconified; if they are, the current
matrices might be invalid by the time we get to using them, because we
should have cleared and recomputed them one or more times.

Anyway, I no longer understand the purpose of this discussion.  I
entered it because I wanted to try to help you understand what you and
Martin were unsure about.  If I succeeded in that, or if I fail to
help you, my job here is done.  If there are still specific situations
or issues that you'd like to understand better, and you think I can
help, please describe those situations/issues in detail, and I will
try to respond to that.

Otherwise, I don't see any point in exchanging single-sentence
arguments that are devoid of any context that I can recognize.  Are we
still talking about the overall way redisplay works, and how it uses
the "garbaged" flag?  Or are we talking about redisplay of a
deiconified frame?  Or maybe we are talking about how iconified frames
should be treated within redisplay?  Or something else entirely?  I
really cannot tell.  The details of each of these use cases are
different, so it is easy to go in circles if we are talking about more
than one situation at a time.

Re: Redisplay problems?

[Stefan]

>> AFAIK iconified frames don't get expose events.
> At least on Windows we have to cater for obscured frames.

But as long as they're obscured, they don't get expose events either,


        Stefan

Re: Redisplay problems?

[Stefan]

>> > This is for visible frames, while we were discussing frames that are
>> > iconified.
>> AFAIK iconified frames don't get expose events.
> Do frames get them when they are deiconified?

If and when needed, they should, yes.  E.g. not if they're deiconified
but obscured by some other window.  And not if they're deiconified and
redrawn directly by the window-system based on a backup of the previous
window's content.  Otherwise, yes.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  kanru@kanru.info,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Sat, 22 Mar 2014 11:42:07 -0400
> 
> >> > This is for visible frames, while we were discussing frames that are
> >> > iconified.
> >> AFAIK iconified frames don't get expose events.
> > Do frames get them when they are deiconified?
> 
> If and when needed, they should, yes.  E.g. not if they're deiconified
> but obscured by some other window.  And not if they're deiconified and
> redrawn directly by the window-system based on a backup of the previous
> window's content.  Otherwise, yes.

When frames that were iconified get the expose event, and the frame's
garbaged flag is not set, we will redraw them using outdated matrices,
because expose_frame uses the current glyph matrices without
recomputing them.

Re: Redisplay problems?

[Stefan]

> When frames that were iconified get the expose event, and the frame's
> garbaged flag is not set, we will redraw them using outdated matrices,
> because expose_frame uses the current glyph matrices without
> recomputing them.

If the current matrices are outdated, then indeed we may expose the
outdated content.  But of course, the next redisplay will fix it, so
it's not terrible.

If the garbaged flag is set, the behavior is not much better: instead of
exposing outdated content we don't expose anything (i.e. it stays
blank), and again the next redisplay should fix.

This said, I think that the more common case of deiconifying/deobscuring
is that the matrices are still up-to-date because nothing has changed in
the mean time.  In that case we're better off not setting the "garbaged"
flag, so we immediately get the right content exposed rather then first
exposing blank.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Sat, 22 Mar 2014 14:43:21 -0400
> 
> > When frames that were iconified get the expose event, and the frame's
> > garbaged flag is not set, we will redraw them using outdated matrices,
> > because expose_frame uses the current glyph matrices without
> > recomputing them.
> 
> If the current matrices are outdated, then indeed we may expose the
> outdated content.  But of course, the next redisplay will fix it, so
> it's not terrible.

You will momentarily show incorrect contents.

> If the garbaged flag is set, the behavior is not much better: instead of
> exposing outdated content we don't expose anything (i.e. it stays
> blank), and again the next redisplay should fix.

It is slightly better, because you never show incorrect contents.

> This said, I think that the more common case of deiconifying/deobscuring
> is that the matrices are still up-to-date because nothing has changed in
> the mean time.  In that case we're better off not setting the "garbaged"
> flag, so we immediately get the right content exposed rather then first
> exposing blank.

I actually don't think we should be bothered about this at all.  Why
does it make sense to optimize the use case where a frame is
deiconified?

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Sat, 22 Mar 2014 14:43:21 -0400
> 
> If the garbaged flag is set, the behavior is not much better: instead of
> exposing outdated content we don't expose anything (i.e. it stays
> blank), and again the next redisplay should fix.

Actually, that's not true: the frame doesn't stay blank.  When the
garbaged flag is set, expose_frame doesn't do anything:

  void
  expose_frame (struct frame *f, int x, int y, int w, int h)
  {
    XRectangle r;
    int mouse_face_overwritten_p = 0;

    TRACE ((stderr, "expose_frame "));

    /* No need to redraw if frame will be redrawn soon.  */
    if (FRAME_GARBAGED_P (f))
      {
	TRACE ((stderr, " garbaged\n"));
	return;
      }

So we never show incorrect or empty contents in that case.

Re: Redisplay problems?

[Stefan]

>> If the garbaged flag is set, the behavior is not much better: instead of
>> exposing outdated content we don't expose anything (i.e. it stays
>> blank), and again the next redisplay should fix.
> Actually, that's not true: the frame doesn't stay blank.  When the
> garbaged flag is set, expose_frame doesn't do anything:

"doesn't do anything" to me means that the screen will be blank.
Or show whatever was displayed there before.  In either case it's not
the correct content.


        Stefan

Re: Redisplay problems?

[Stefan]

> I actually don't think we should be bothered about this at all.  Why
> does it make sense to optimize the use case where a frame is
> deiconified?

If you have 50 frames, 25 on one desktop and 25 on the other, whenever
you switch from one desktop to the other, 25 frames get deiconified and
the other 25 get iconified.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Sun, 23 Mar 2014 21:58:32 -0400
> 
> > I actually don't think we should be bothered about this at all.  Why
> > does it make sense to optimize the use case where a frame is
> > deiconified?
> 
> If you have 50 frames, 25 on one desktop and 25 on the other, whenever
> you switch from one desktop to the other, 25 frames get deiconified and
> the other 25 get iconified.

I still don't see anything performance critical even in this scenario.
A switch to a different desktop is not something one would do several
times a second, and it's okay for it to take a second or so.

Re: Redisplay problems?

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan <monnier@IRO.UMontreal.CA>
>> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>>         emacs-devel@gnu.org
>> Date: Sun, 23 Mar 2014 21:58:32 -0400
>> 
>> > I actually don't think we should be bothered about this at all.  Why
>> > does it make sense to optimize the use case where a frame is
>> > deiconified?
>> 
>> If you have 50 frames, 25 on one desktop and 25 on the other, whenever
>> you switch from one desktop to the other, 25 frames get deiconified and
>> the other 25 get iconified.
>
> I still don't see anything performance critical even in this scenario.
> A switch to a different desktop is not something one would do several
> times a second, and it's okay for it to take a second or so.

It is quite customary to _cycle_ through desktops and go through several
in fast succession until finding the desired one.  Most certainly at a
rate of several (three or four) per second.

-- 
David Kastrup

Re: Redisplay problems?

[Stefan]

> I still don't see anything performance critical even in this scenario.

I lived with the older slower behavior for many years, indeed.

But computers aren't getting very much faster and avoiding a rebuild of
the glyph matrices sped up some of my use cases significantly, making
the behavior noticeably smoother.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Mon, 24 Mar 2014 09:32:20 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Stefan <monnier@IRO.UMontreal.CA>
> >> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
> >>         emacs-devel@gnu.org
> >> Date: Sun, 23 Mar 2014 21:58:32 -0400
> >> 
> >> > I actually don't think we should be bothered about this at all.  Why
> >> > does it make sense to optimize the use case where a frame is
> >> > deiconified?
> >> 
> >> If you have 50 frames, 25 on one desktop and 25 on the other, whenever
> >> you switch from one desktop to the other, 25 frames get deiconified and
> >> the other 25 get iconified.
> >
> > I still don't see anything performance critical even in this scenario.
> > A switch to a different desktop is not something one would do several
> > times a second, and it's okay for it to take a second or so.
> 
> It is quite customary to _cycle_ through desktops and go through several
> in fast succession until finding the desired one.  Most certainly at a
> rate of several (three or four) per second.

You took what I wrote too literally, I think.  What you describe
deiconifies frames at higher rate, but only momentarily so.  Once the
desired desktop is located, the user will stay with it for some time,
until she again switches.  So I still see no reason to regard this as
performance critical.

OTOH, when we need to deiconify a frame, or expose a large portion of
it, I see no space for significant optimizations anyway.  Redisplay
optimizations are about redrawing as small portions of the frame as
possible.  But in the case in point, we basically need to redraw the
entire frame -- how much can you win here anyway?

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Mon, 24 Mar 2014 08:33:30 -0400
> 
> > I still don't see anything performance critical even in this scenario.
> 
> I lived with the older slower behavior for many years, indeed.
> 
> But computers aren't getting very much faster and avoiding a rebuild of
> the glyph matrices sped up some of my use cases significantly, making
> the behavior noticeably smoother.

On these general terms, we are in violent agreement.  (Btw, what about
parallel redisplaying of several frames by several independent
threads?)

However, what I wrote was only about deiconifying a frame, or
redrawing a frame which was entirely obscured.  If we are still
talking about that case, please tell more about the situations where
deiconifying a frame was a bottleneck or even a significant annoyance.

In any case, I'm not opposed to attempts to optimize these scenarios.
How to do that is another question.  Blindly assuming that the current
matrices are up to date doesn't sound like a good idea to me: we will
momentarily flash incorrect display, and sooner or later people will
complain.  And such an incorrect redisplay will be immediately
followed another one, so what exactly have we gained?  But we could do
this when the redisplay flag of the frame is not set, for example.

If the redisplay flag of an obscured/iconified frame _is_ set, then it
looks like you cannot win anyway, because the entire frame needs to be
completely redisplayed, and comparing against the current matrices is
meaningless (since nothing is on the glass, so a full redraw is the
only alternative).  So I think setting the garbaged flag in this case
is TRT.

IOW, the garbaged flag of an obscured or iconified frame should only
be set after its redisplay flag is set.  I don't see any place for
further optimization; do you?

Re: Redisplay problems?

[Stefan]

> (Btw, what about parallel redisplaying of several frames by several
> independent threads?)

Patch welcome.

> However, what I wrote was only about deiconifying a frame, or
> redrawing a frame which was entirely obscured.  If we are still
> talking about that case, please tell more about the situations where
> deiconifying a frame was a bottleneck or even a significant annoyance.

Occasionally, closing a large frame that obscured a dozen frames
underneath could be such a problem, but since in X11 we don't optimize
"obscured" like we do under w32 I haven't noticed it.
In any case, Emacs can't tell the difference between "switching
workspace" and "deiconifying", AFAIK.

> How to do that is another question.  Blindly assuming that the current
> matrices are up to date doesn't sound like a good idea to me: we will
> momentarily flash incorrect display, and sooner or later people will
> complain.  And such an incorrect redisplay will be immediately
> followed another one, so what exactly have we gained?

In case the next redisplay is far away, it's better to display the old
window's content than to leave "garbage" on the screen, I think.
I don't think it's a very serious issue either way.

But the code had a comment specifically mentioning that we preferred
not setting the `garbaged' flag (when possible) so as to avoid
a blank/undraw frame being displayed if a frame is deiconified while
we're in the middle of a toolkit menu.

> But we could do this when the redisplay flag of the frame is not set,
> for example.

We could, yes.  But I'm not sure it would really be better.

> If the redisplay flag of an obscured/iconified frame _is_ set, then it
> looks like you cannot win anyway, because the entire frame needs to be
> completely redisplayed, and comparing against the current matrices is
> meaningless (since nothing is on the glass, so a full redraw is the
> only alternative).  So I think setting the garbaged flag in this case
> is TRT.

No: setting the garbaged flag has the side-effect of disabling all
optimizations while computing the new matrices.  And in this case, we
don't need to disable those optimizations, AFAIK.


        Stefan

Re: Redisplay problems?

[Stefan]

> OTOH, when we need to deiconify a frame, or expose a large portion of
> it, I see no space for significant optimizations anyway.  Redisplay
> optimizations are about redrawing as small portions of the frame as
> possible.  But in the case in point, we basically need to redraw the
> entire frame -- how much can you win here anyway?

Of course, the window-system may elect to do some of this redraw on its
own by keeping a copy of the frame's last content (under X11 the
corresponding feature is called "BackingStore").  In that case we don't
need to redraw the entire frame.  We may often not need to redraw
anything at all.

IIRC this "BackingStore" option is typically disabled nowadays in X11,
so we shouldn't pay too much attention to it.  But I'm not sure how
compositing window-managers behave in this respect, since they kind of
"naturally" have a kinf of backing-store.  IOW, if we want to be serious
about this discussion, we should first get some real data from w32,
Gnome, and ns cases to see if in practice, a deiconifiy is almost always
followed by an "expose" of the frame, or not.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Mon, 24 Mar 2014 14:07:34 -0400
> 
> > If the redisplay flag of an obscured/iconified frame _is_ set, then it
> > looks like you cannot win anyway, because the entire frame needs to be
> > completely redisplayed, and comparing against the current matrices is
> > meaningless (since nothing is on the glass, so a full redraw is the
> > only alternative).  So I think setting the garbaged flag in this case
> > is TRT.
> 
> No: setting the garbaged flag has the side-effect of disabling all
> optimizations while computing the new matrices.  And in this case, we
> don't need to disable those optimizations, AFAIK.

Without anything on the screen that reflects the current matrices,
what would be the point of these optimizations?  These optimizations
only make sense when portions of the frame are already on the glass,
because we can then avoid both recomputing and redrawing those
portions, or some of them.  But when nothing is on the glass, these
optimizations will not help, because you must redraw everything, and
AFAIK we don't currently have a redisplay mode where the portions of
matrices that were not recomputed are nevertheless redrawn.

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@IRO.UMontreal.CA>
> Cc: David Kastrup <dak@gnu.org>, emacs-devel@gnu.org
> Date: Mon, 24 Mar 2014 14:15:00 -0400
> 
> > OTOH, when we need to deiconify a frame, or expose a large portion of
> > it, I see no space for significant optimizations anyway.  Redisplay
> > optimizations are about redrawing as small portions of the frame as
> > possible.  But in the case in point, we basically need to redraw the
> > entire frame -- how much can you win here anyway?
> 
> Of course, the window-system may elect to do some of this redraw on its
> own by keeping a copy of the frame's last content (under X11 the
> corresponding feature is called "BackingStore").  In that case we don't
> need to redraw the entire frame.  We may often not need to redraw
> anything at all.

Not redrawing anything is easy, provided that we know we can.

> IIRC this "BackingStore" option is typically disabled nowadays in X11,
> so we shouldn't pay too much attention to it.  But I'm not sure how
> compositing window-managers behave in this respect, since they kind of
> "naturally" have a kinf of backing-store.  IOW, if we want to be serious
> about this discussion, we should first get some real data from w32,
> Gnome, and ns cases to see if in practice, a deiconifiy is almost always
> followed by an "expose" of the frame, or not.

I already looked on w32: when a frame is deiconified, we always get a
WM_PAINT message telling us to repaint the whole frame, which I guess
is the Windows equivalent of the Expose event.

Re: Redisplay problems?

[Stefan]

> optimizations will not help, because you must redraw everything, and
> AFAIK we don't currently have a redisplay mode where the portions of
> matrices that were not recomputed are nevertheless redrawn.

They'll be redrawn from the current matrices in response to the
expose events.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Mon, 24 Mar 2014 16:43:03 -0400
> 
> > optimizations will not help, because you must redraw everything, and
> > AFAIK we don't currently have a redisplay mode where the portions of
> > matrices that were not recomputed are nevertheless redrawn.
> 
> They'll be redrawn from the current matrices in response to the
> expose events.

Only if you insist on showing incorrect contents first, and if you can
arrange for the expose event to always be processed before the
redisplay.

Re: Redisplay problems?

[Stefan Monnier]

>> > optimizations will not help, because you must redraw everything, and
>> > AFAIK we don't currently have a redisplay mode where the portions of
>> > matrices that were not recomputed are nevertheless redrawn.
>> They'll be redrawn from the current matrices in response to the
>> expose events.
> Only if you insist on showing incorrect contents first, and if you can
> arrange for the expose event to always be processed before the
> redisplay.

Hmm... I guess it depends on what kind of drawing operations we use.
E.g. if the redisplay optimizations include usage of "block move"
operations, then indeed, if the redisplay happens before the expose, we
may end up moving not-yet-drawn (i.e. incorrect/garbage) pixels.

Do we use things like "block move"?


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Tue, 25 Mar 2014 09:10:44 -0400
> 
> Hmm... I guess it depends on what kind of drawing operations we use.
> E.g. if the redisplay optimizations include usage of "block move"
> operations, then indeed, if the redisplay happens before the expose, we
> may end up moving not-yet-drawn (i.e. incorrect/garbage) pixels.
> 
> Do we use things like "block move"?

Not sure what you mean by "block move".  From the name and the
context, it sounds like you are talking about actual drawing, i.e. on
a level that underlies the display back-ends in xterm.c, w32term.c
etc.  If so, that's way below the level of redisplay optimizations
that are in xdisp.c and even in dispnew.c, which are both
device-independent.

But maybe I misunderstand what you mean.

RE: Redisplay problems?

[Drew Adams]

Dunno whether this is pertinent to this thread. If not, ignore.

It just happened for me that (a) I was in an Emacs (recent build)
buffer/frame and then (b) I clicked a non-Emacs window mgr window.
(Neither the Emacs frame nor the other window-mgr window obscured
the other.  They were just next to each other on the screen,
without overlapping.)

The Emacs frame display immediately turned blank - lost *all*
display of the text etc.  I clicked back in the Emacs frame - still
nothing displayed.  I hit `C-l', and only a little bit of the text
was then displayed.

This didn't happen previously, AFAIK.  (I naively get the
impression that someone has being trying too hard to skip some
redisplaying, in an attempt to optimize things.  Too clever by
half, perhaps?)

Again, if not helpful at all, please ignore.

Re: Redisplay problems?

[Stefan Monnier]

>> Do we use things like "block move"?
> Not sure what you mean by "block move".

Moving a block of pixels.
My concern is whether the redraw may decide to use some other part of
the display to draw a particular region (typically for scrolling).
If we do, then it's important to make sure that part is in-sync with the
current matrices, but if we don't then we don't need to worry about the
display being out-of-sync with the current matrices.


        Stefan

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: rudalics@gmx.at, christian@defun.dk, kanru@kanru.info, cloos@jhcloos.com,
>         emacs-devel@gnu.org
> Date: Thu, 27 Mar 2014 09:55:40 -0400
> 
> >> Do we use things like "block move"?
> > Not sure what you mean by "block move".
> 
> Moving a block of pixels.

Got that part, thanks.

> My concern is whether the redraw may decide to use some other part of
> the display to draw a particular region (typically for scrolling).
> If we do, then it's important to make sure that part is in-sync with the
> current matrices, but if we don't then we don't need to worry about the
> display being out-of-sync with the current matrices.

Here you lost me.  Are we still talking about redrawing a frame as
response to expose event that exposes the entire frame?  If so,
expose_frame doesn't bother to check whether the current matrix is up
to date, it blindly uses all of it to redraw every screen line of the
frame.

In case of text lines, this redraws each screen line by calling the
'draw' method of the font driver, passing it a "glyph string"
structure, which describes a sequence of characters that come from the
same face (font. colors, etc.).

How does all this relate to moving blocks of pixels?

Re: Redisplay problems?

[Stefan Monnier]

> How does all this relate to moving blocks of pixels?

0 - frame gets deiconified, we don't redraw anything.
1 - we recompute desired matrices.
2 - we compare matrices and see that the top part of the frame can be drawn
    be copying the pixels from the bottom part (assuming here incorrectly
    that those pixels are in sync with the current matrices).
3 - we copy the incorrect pixels and consider ourselves happy to have
    redisplayed the top-part of the frame :-(

I think this can't happen, which is why I think it's OK not to redraw
anything at step 0.


        Stefan

Re: Redisplay problems?

[Christian Lynbech]

I can confirm that the redisplay issues I have been seeing have
disappeared, both on Linux and OSX.

------------------------+-----------------------------------------------------
Christian Lynbech       | christian #\@ defun #\. dk
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
                                        - petonic@hal.com (Michael A. Petonic)

Re: Redisplay problems?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rudalics@gmx.at,  christian@defun.dk,  kanru@kanru.info,  cloos@jhcloos.com,  emacs-devel@gnu.org
> Date: Thu, 27 Mar 2014 17:13:00 -0400
> 
> > How does all this relate to moving blocks of pixels?
> 
> 0 - frame gets deiconified, we don't redraw anything.
> 1 - we recompute desired matrices.
> 2 - we compare matrices and see that the top part of the frame can be drawn
>     be copying the pixels from the bottom part (assuming here incorrectly
>     that those pixels are in sync with the current matrices).
> 3 - we copy the incorrect pixels and consider ourselves happy to have
>     redisplayed the top-part of the frame :-(
> 
> I think this can't happen, which is why I think it's OK not to redraw
> anything at step 0.

I think it can happen.  See scrolling_window and x_scroll_run that it
calls.