bug#27668: 26.0.50; Crash with display-line-numbers t

[Eli Zaretskii <eliz@gnu.org>]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Fri, 14 Jul 2017 13:45:04 +0200
> 
> >       /* Try to redisplay starting at same place as before.
> >          If point has not moved off frame, accept the results.  */
> >       if (!current_matrix_up_to_date_p
> > 	  /* Don't use try_window_reusing_current_matrix in this case
> > 	     because a window scroll function can have changed the
> > 	     buffer.  */
> > 	  || !NILP (Vwindow_scroll_functions)
> > 	  || MINI_WINDOW_P (w)
> > 	  || !(used_current_matrix_p
> > 	       = try_window_reusing_current_matrix (w)))
> > 	{
> > 	  IF_DEBUG (debug_method_add (w, "1"));
> > 	  if (try_window (window, startp, TRY_WINDOW_CHECK_MARGINS) < 0)
> >
> > It cannot be MINI_WINDOW_P, so it's either current_matrix_up_to_date_p
> > is false, or your window-scroll-functions is non-nil, or we called
> > try_window_reusing_current_matrix.  In the latter case, I'd expect my
> > recent change to fix the problem, so I guess that function wasn't
> > called, and some other condition caused us to call try_window.  Or
> > maybe I'm missing something here.
> >
> 
> (gdb) p used_current_matrix_p
> $1 = false
> (gdb) p Vwindow_scroll_functions
> $2 = XIL(0)
> (gdb) p w
> $3 = <optimized out>
> (gdb) p current_matrix_up_to_date_p
> $4 = false

So the trigger is current_matrix_up_to_date_p, which is false.  That
explains why my recent changes didn't solve the problem: the function
where I made those changes wasn't called.

> Would you like me to try an unoptimised build? CFLAGS=-O0 -ggdb or
> similar?

It cannot hurt, so please do.  Maybe this will give some valuable
hints.  (Please use "-gdwarf-4 -g3" instead of -ggdb, it should
provide a better debug info.)  Btw, I already did an optimized build,
but was unable to reproduce the problem there as well.

> Basically, I have a buffer where display-line-numbers is nil, then I
> either switch to a buffer where it's t or visit a file where the
> mode-hook sets it to t. In this case I was looking at a diff hunk,
> where Magit does all sorts of highlighting and font-locking, and it
> visits the underlying file for you when you hit RET
> 
> > Did the previous times also happened when switching from a Magit
> > buffer to a buffer under display-line-numbers?
> 
> No, previous times were when calling C-x C-f from a buffer with
> display-line-numbers t

That's what I tried, but the problem didn't happen.

When the assertion in maybe_produce_line_number is hit, what are the
values of it->vpos and it->glyph_row->y?  Are they always the same
values?  If they are, maybe we could put a watchpoint on the
corresponding glyph row and see who changes it.

The problem seems to be that display_line starts producing glyphs in a
glyph row which wasn't cleared, i.e. its used[1] counter is non-zero.
The call to prepare_desired_row at the beginning of display_line is
supposed to do that, but only if the row->enabled_p flag is reset.
This flag should be reset for all the glyph rows of the window's
desired_matrix, because redisplay calls clear_glyph_matrix for
w->desired_matrix, directly and indirectly, in many places.  Somehow
in your case either those calls to clear_glyph_matrix are bypassed or
some code sets the enabled_p flag at some point and doesn't reset it
before the call to try_window on line 16991 of xdisp.c.  I'm trying to
establish where does this happen and why.

Just to make sure I'm on the right track: if you make the change
below, does the problem go away?

diff --git a/src/xdisp.c b/src/xdisp.c
index 2aceb89..341a1e3 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -16988,6 +16988,7 @@ redisplay_window (Lisp_Object window, bool just_this_one_p)
 	       = try_window_reusing_current_matrix (w)))
 	{
 	  IF_DEBUG (debug_method_add (w, "1"));
+	  clear_glyph_matrix (w->desired_matrix);
 	  if (try_window (window, startp, TRY_WINDOW_CHECK_MARGINS) < 0)
 	    /* -1 means we need to scroll.
 	       0 means we need new matrices, but fonts_changed