bug#13864: 24.3.50; emacsclient -t loops when connected to emacs server running in X11

[Eli Zaretskii <eliz@gnu.org>]

> From: ashish.is@lostca.se (Ashish SHUKLA)
> Cc: 13864@debbugs.gnu.org
> Date: Wed, 13 Mar 2013 14:30:05 +0530
> 
> Sorry for the delay in follow-up.

No need to apologize, we all have our lives.

> No, it only starts flickering after I focus to X11 window and focus back to
> emacsclient xterm window. Apologies, if I wasn't clear before in my
> observations.

Please make it flicker for collecting the data I describe below, it is
very important for me to be sure that the data is relevant.

> > So you are saying that scrolling_1 is never called, is that right?
> 
> Right.

OK, that makes the list of suspects quite a bit shorter.

> >   . I see tooltip messages being displayed in the echo area.  You have
> >     a mouse active (as far as Emacs is concerned) on the xterm frame,
> >     is that right?  Can you disable it and see if the flickering
> >     persists?
> 
> I don't know what you meant by mouse active. FTR, I don't use
> "xterm-mouse-mode" in my .emacs.d/init.el nor the -Q config has that, if
> that's what you're implying. Emacs instance in xterm doesn't have any effect
> of mouse in it. The tooltip is courtesy some spurious key-presses during
> debugging..

That's strange, I'm probably missing something.  Not terribly
important (it's tangential to the issue I'm hunting with your GDB
collected data), but could you give me a recipe to cause such a
tooltip in the xterm frame by some key-press?

> >   (gdb) break update_frame_line
> >   (gdb) commands
> >> p vpos
> >> continue
> >> end
> >   (gdb)
> 
> > Please see if you see all the line numbers when you recreate the
> > situation with flickering.
> 
> Yes, I saw them, the output is in "gdb-1.txt" attachment. The GDB output has
> inline comments prefixed with '===='.

OK, that means Emacs tries to redraw the entire frame, line by line.

> >   if (!FRAME_CHAR_INS_DEL_OK (f))
> >     {
> >       int i, j;
> 
> >       /* Find the first glyph in desired row that doesn't agree with
> > 	 a glyph in the current row, and write the rest from there on.  */
> >       for (i = 0; i < nlen; i++)
> > 	{
> > 	  if (i >= olen || !GLYPH_EQUAL_P (nbody + i, obody + i))
> > 	    {
> > 	      /* Find the end of the run of different glyphs.  */
> > 	      j = i + 1;
> > 	      while (j < nlen
> > 		     && (j >= olen
> > 			 || !GLYPH_EQUAL_P (nbody + j, obody + j)
> > 			 || CHAR_GLYPH_PADDING_P (nbody[j])))
> > 		++j;
> 
> > 	      /* Output this run of non-matching chars.  */
> > 	      cursor_to (f, vpos, i);
> > 	      write_glyphs (f, nbody + i, j - i);
> > 	      i = j - 1;
> 
> > 	      /* Now find the next non-match.  */
> > 	    }
> > 	}
> 
> >       /* Clear the rest of the line, or the non-clear part of it.  */
> >       if (olen > nlen)
> > 	{
> > 	  cursor_to (f, vpos, nlen);
> > 	  clear_end_of_line (f, olen);
> > 	}
> 
> >       /* Make current row = desired row.  */
> >       make_current (desired_matrix, current_matrix, vpos);
> >       return;  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
> >     }
> 
> > on the marked line, and see if it ever breaks.
> 
> This output is in "gdb-2.txt" with inline comments.

I see that breakpoint never breaks, which would mean
FRAME_CHAR_INS_DEL_OK returns non-zero on your xterm, as expected.
Again, this trims the list of suspects.

So now the problem description is this:

 . update_frame_line is being called for every line of the frame.  I
   don't yet know why, nor whether this is a bug or not, but it's a
   separate issue anyway.

 . There's code in update_frame_line that attempts to avoid redrawing
   the portions of display that are already up to date, i.e. that are
   unchanged since the last redisplay cycle.  The flickering and your
   truss output indicate that significant portions of the display are
   being redrawn nonetheless.  The question is, why?  What code in
   update_frame_line fails to detect that most or all of the display
   did not change at all, and why?

To answer the last questions, please use the following GDB setup.
(Please verify the line numbers before you set each breakpoint, in
case your sources are slightly different from what I'm using to write
this message.)

 (gdb) break dispnew.c:4845
 (gdb) commands
  > p vpos
  > p desired_row->used[1]
  > p nlen
  > continue
  > end

This puts a breakpoint on this line:

      /* Write the contents of the desired line.  */
      if (nlen)
	{
          cursor_to (f, vpos, 0);
	  write_glyphs (f, nbody, nlen);  <<<<<<<<<<<<<<<<<<<<<<<
	}

 (gdb) break dispnew.c:4854
 (gdb) commands
  > p vpos
  > p f->total_cols  
  > p nlen
  > continue
  > end
 (gdb) break dispnew.c:4859
 (gdb) commands
  > p vpos
  > continue
  > end

These 2 breakpoints are here:

      if (nlen < FRAME_TOTAL_COLS (f))
	{
	  cursor_to (f, vpos, nlen);
          clear_end_of_line (f, FRAME_TOTAL_COLS (f));  <<<<<<<<<<<<<<<
	}
      else
	/* Make sure we are in the right row, otherwise cursor movement
	   with cmgoto might use `ch' in the wrong row.  */
	cursor_to (f, vpos, 0);  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

 (gdb) break dispnew.c:4926
 (gdb) commands
  > p vpos
  > p desired_row->used[1]
  > p nlen
  > p nsp
  > continue
  > end

This breakpoint is here:

      if (nlen > nsp)
	{
	  cursor_to (f, vpos, nsp);
	  write_glyphs (f, nbody + nsp, nlen - nsp);  <<<<<<<<<<<<<<<<<<
	}

 (gdb) break dispnew.c:4999
 (gdb) commands
  > p vpos
  > p olen
  > p osp
  > p desired_row->used[1]
  > p nsp
  > continue
  > end

This is here:

  /* Now go through the line, inserting, writing and
     deleting as appropriate.  */

  if (osp > nsp)
    {
      cursor_to (f, vpos, nsp);
      delete_glyphs (f, osp - nsp);  <<<<<<<<<<<<<<<<<<<<<<
    }

 (gdb) break dispnew.c:5006
 (gdb) commands
  > p vpos
  > p nsp
  > p osp
  > p begmatch
  > p endmatch
  > p olen
  > p nlen
  > continue
  > end

This breakpoint is here:

  else if (nsp > osp)
    {
      /* If going to delete chars later in line
	 and insert earlier in the line,
	 must delete first to avoid losing data in the insert */
      if (endmatch && nlen < olen + nsp - osp)  <<<<<<<<<<<<<<<<<<<<<<
	{
	  cursor_to (f, vpos, nlen - endmatch + osp - nsp);
	  delete_glyphs (f, olen + nsp - osp - nlen);
	  olen = nlen - (nsp - osp);
	}

 (gdb) break dispnew.c:5035
 (gdb) commands
  > p vpos
  > p nsp
  > p osp
  > p begmatch
  > p endmatch
  > p olen
  > p nlen
  > p desired_row->used[1]
  > p tem
  > continue
  > end

This puts a breakpoint here:

	  /* Function write_glyphs is prepared to do nothing
	     if passed a length <= 0.  Check it here to avoid
	     unnecessary cursor movement.  */
	  if (nlen - tem > 0)
	    {
	      cursor_to (f, vpos, nsp + begmatch);
	      write_glyphs (f, nbody + nsp + begmatch, nlen - tem); <<<<<<<
	    }

Two more breakpoints with similar commands:

 (gdb) break dispnew.c:5063
 (gdb) commands
  > p vpos
  > p nsp
  > p osp
  > p begmatch
  > p endmatch
  > p olen
  > p nlen
  > p desired_row->used[1]
  > p tem
  > p del
  > p out
  > continue
  > end
 (gdb) break dispnew.c:5069
 (gdb) commands
  > p vpos
  > p nsp
  > p osp
  > p begmatch
  > p endmatch
  > p olen
  > p nlen
  > p desired_row->used[1]
  > p tem
  > continue
  > end

They are here:


	  /* If we left columns to be overwritten, we must delete them.  */
	  del = olen - tem - out;
	  if (del > 0)
	    delete_glyphs (f, del);

	  /* At last, we insert columns not yet written out.  */
	  insert_glyphs (f, nbody + nsp + begmatch + out, nlen - olen + del); <<<<<<<<<<<
	  olen = nlen;
	}
      else if (olen > nlen)
	{
	  cursor_to (f, vpos, nsp + begmatch);
	  write_glyphs (f, nbody + nsp + begmatch, nlen - tem); <<<<<<<<<<<
	  delete_glyphs (f, olen - nlen);
	  olen = nlen;
	}

And the last one:

 (gdb) break dispnew.c:5080
 (gdb) commands
  > p vpos
  > p olen
  > p nlen
  > p desired_row->used[1]
  > p desired_row->enabled_p
  > continue
  > end

This breakpoint is here:

 just_erase:
  /* If any unerased characters remain after the new line, erase them.  */
  if (olen > nlen)
    {
      cursor_to (f, vpos, nlen);
      clear_end_of_line (f, olen); <<<<<<<<<<<<<<<<<<<
    }

This is certainly a lot of typing, so I suggest to put it all (sans
the "(gdb)" and ">" parts) in a file, and then "source that-file" from
inside GDB.  This way, you will be able to repeat the experiment
without going through the pain of retyping it all again.  (Don't
forget adding to that file "set logging on" and a breakpoint on
Fredraw_display.)

Once you are set up in GDB, make Emacs flicker, and collect the data
printed by GDB.  The goal of these breakpoints is to see which code is
involved in the flickering situation, and which parts of it are
actually writing to the screen.

Thanks.