Debugging printing to stderr prior glyph row values ....

Debugging printing to stderr prior glyph row values ....

[Keith David Bershatsky]

I am working on feature requests 22873 (multiple fake cursors) and 17684 (crosshairs that track the cursor position).

I am having trouble preventing Emacs from occasionally crashing when printing messages to stderr while update_window is occurring, which makes debugging other issues more difficult than necessary.

update_window processes one glyph row at a time using the w->desired_matrix; and, feature requests 22873/17684 use both the desired/current matrices.

My best guess is that the problem lies here:

(gdb) print previous_row__last_glyph
$9 = (struct glyph *) 0x11f7fffb0

(gdb) print previous_row__last_glyph->charpos
Cannot access memory at address 0x11f7fffd0

Some suggestions regarding how to prevent Emacs from crashing would be greatly appreciated.  A full bt of frame 0 follows at the end of this post.

  struct glyph *previous_row__penultimate_glyph
    = (vpos == 0)
        ? NULL
      : (!(row - 1)->enabled_p)
        ? NULL
      : ((row - 1)->used[TEXT_AREA] > 1)
        ? (row - 1)->glyphs[TEXT_AREA] + (row - 1)->used[TEXT_AREA] - 2
      : NULL;

  struct glyph *previous_row__last_glyph
    = (vpos == 0)
        ? NULL
      : (!(row - 1)->enabled_p)
        ? NULL
      : (row - 1)->glyphs[TEXT_AREA] + (row - 1)->used[TEXT_AREA] - 1;

  struct glyph *penultimate_glyph
    = (row->used[TEXT_AREA] < 1)
      ? row->glyphs[TEXT_AREA] + row->used[TEXT_AREA] - 2
      : NULL;

  struct glyph *last_glyph
    = row->glyphs[TEXT_AREA] + row->used[TEXT_AREA] - 1;

  fprintf (stderr, "\nA.  vpos (%d) | glyph->charpos (%d) | posint (%d) | mc_zv (%d) | mc_zv_byte (%d)\n\
  row - 1:  penultimate_charpos (%d) | last_charpos (%d) | ends_at_zv_p (%s)\n\
  row - 1:  start/end_bytepos (%d/%d) | start/end_charpos (%d/%d)\n\
  row_at_zv_p (%s) | row_beyond_zv_p (%s)\n\
  row:  penultimate_charpos (%d) | last_charpos (%d) | ends_at_zv_p (%s)\n\
  row:  start/end_bytepos (%d/%d) | start/end_charpos (%d/%d)\n",
    vpos, start->charpos, posint, mc_zv, mc_zv_byte,
    previous_row__penultimate_glyph != NULL
      ? previous_row__penultimate_glyph->charpos
      : -1,
    previous_row__last_glyph != NULL
      ? previous_row__last_glyph->charpos
      : -1,
      (row - 1)->ends_at_zv_p ? "y" : "n",
    MATRIX_ROW_START_BYTEPOS (row - 1), MATRIX_ROW_END_BYTEPOS (row - 1),
      MATRIX_ROW_START_CHARPOS (row - 1), MATRIX_ROW_END_CHARPOS (row - 1),
    row_at_zv_p ? "y" : "n", row_beyond_zv_p ? "y" : "n",
    penultimate_glyph != NULL
      ? penultimate_glyph->charpos
      : -1,
    last_glyph->charpos, row->ends_at_zv_p ? "y" : "n",
    MATRIX_ROW_START_BYTEPOS (row), MATRIX_ROW_END_BYTEPOS (row),
      MATRIX_ROW_START_CHARPOS (row), MATRIX_ROW_END_CHARPOS (row));

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(gdb) print previous_row__penultimate_glyph
$8 = (struct glyph *) 0x0

(gdb) print previous_row__last_glyph
$9 = (struct glyph *) 0x11f7fffb0

(gdb) print previous_row__last_glyph->charpos
Cannot access memory at address 0x11f7fffd0

(gdb) print *previous_row__last_glyph
Cannot access memory at address 0x11f7fffb0

(gdb) print last_glyph
$10 = (struct glyph *) 0x11fa46200

(gdb) print last_glyph->charpos
$11 = -1

(gdb) print penultimate_glyph
$12 = (struct glyph *) 0x0

(gdb) bt full

#0  0x0000000100045d1b in mc_draw_row (w=0x1088ef048, matrix=0x106ddb0a0, 
    row=0x11d1b2e00, start=0x11fa46200, x=0, hpos_length=1, vpos=8, 
    bottom_vpos=113, active_p=true, cursor_matrix=0x106be6580, fc_x=770, 
    ch_foreground=..., fc_foreground=..., mc_zv=16, mc_zv_byte=16, 
    draw_p=true, from_where=SCRIBE_ONE) at xdisp.c:4604
        hpos = 0
        previous_glyph = 0x0
        skip_p = false
        posint = -1
        specs = {
          i = 0
        }
        hpos_start = 0
        window = {
          i = 4438552653
        }
        f = 0x10388f1a0
        buf = {
          i = 4354225669
        }
        b = 0x103883600
        opoint_fx = 1
        opoint_fy = 143585352
        opoint_wd = 1
        opoint_h = 488320512
        text_area_width = 975
        header_line_format = 24
        previous_row__penultimate_glyph = 0x0
        previous_row__last_glyph = 0x11f7fffb0
        penultimate_glyph = 0x0
        last_glyph = 0x11fa46200
        row_at_zv_p = false
        row_beyond_zv_p = true
        cursor_row = 0x11fa17100
        cursor_glyph = 0x11fa236b0
        cursor_at_fringe_p = false
        cursor_beyond_fringe_p = false
        bg_face_id = DEFAULT_FACE_ID
        cursor_bg = {
          red = 0, 
          green = 0, 
          blue = 0
        }
        lisp_type = {
          i = 24336
        }
        cursor_type = MC_HOLLOW_BOX
        cursor_width = -1
        cursor_face_id = 36
        cursor_face = 0x100f0fc30
        cursor_color = {
          i = 4299532436
        }
        cursor_fg = {
          red = 0, 
          green = 1, 
          blue = 1
        }
        fill_column_trumps_p = false

Re: Debugging printing to stderr prior glyph row values ....

[Eli Zaretskii]

> Date: Wed, 22 May 2019 21:14:01 -0700
> From: Keith David Bershatsky <esq@lawlist.com>
> 
> (gdb) print previous_row__last_glyph
> $9 = (struct glyph *) 0x11f7fffb0
> 
> (gdb) print previous_row__last_glyph->charpos
> Cannot access memory at address 0x11f7fffd0

You didn't tell enough about the situation.  E.g., what kind of window
is that -- a window that shows a buffer, or something else?  How many
rows are in that window?  What is the full content or ROW and ROW-1?

One potential problem I see is that this doesn't test the 'used'
member:

  struct glyph *previous_row__last_glyph
    = (vpos == 0)
        ? NULL
      : (!(row - 1)->enabled_p)
        ? NULL
      : (row - 1)->glyphs[TEXT_AREA] + (row - 1)->used[TEXT_AREA] - 1;

The 'used' member should be positive, or else subtracting 1 is not
allowed.

Re: Debugging printing to stderr prior glyph row values ....

[Keith David Bershatsky]

Thank you, Eli, for having a look at this particular thread.

I will add a test for the `used` row member to ensure that it is positive.  The crashing reported in the initial post only happens somtimes, so I'll need to do some further testing.

The current issue relates to a window displaying a buffer, and the leading/introductory function used to draw fake cursors has a check at the outset:

  if (!BUFFERP (w->contents))
    return;

I am looking to programmatically distinguish between the following three types of consecutive rows:  (1) the penultimate row _not_ ending in ZV that immediately precedes a row with _only_ ZV; (2) the last row that contains _only_ ZV; and, (3) rows that follow a row with _only_ ZV.

  Every good boy deserves fudge.¶
  ZV<==
  [one or more ' ' space glyphs]
  [one or more ' ' space glyphs]
  Mode-Line

Fake cursors are drawn immediately after each section [HPOS + LEN] of a potentially updated row in the desired_matrix is processed by update_window; and, then fake cursors are drawn on _other_ rows using the current_matrix (after all potentially updated rows have been processed using the desired_matrix).

Situation number 2 is labeled below as row_at_zv_p; and, stuation number 3 is labeled below as row_beyond_zv_p.  The problem I forsee with the row_at_zv_p test is that there is no guarantee that ROW - 1 will be enabled in the MATRIX containing ROW, and the test is therefore unreliable.

  bool row_at_zv_p = (MATRIX_ROW_START_BYTEPOS (row - 1) != MATRIX_ROW_END_BYTEPOS (row - 1)
                      && !(row - 1)->ends_at_zv_p
                      && MATRIX_ROW_START_BYTEPOS (row) == MATRIX_ROW_END_BYTEPOS (row)
                      && row->ends_at_zv_p);

  bool row_beyond_zv_p = (MATRIX_ROW_START_BYTEPOS (row) == MATRIX_ROW_END_BYTEPOS (row));

A better method of determining row_at_zv_p (without consulting ROW - 1) would be appreciated, if such a test exists.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [05-23-2019 07:43:14] <23 May 2019 17:43:14 +0300>
> From: Eli Zaretskii <eliz@gnu.org>
> 
> * * *
> 
> You didn't tell enough about the situation.  E.g., what kind of window
> is that -- a window that shows a buffer, or something else?  How many
> rows are in that window?  What is the full content or ROW and ROW-1?
> 
> One potential problem I see is that this doesn't test the 'used'
> member:
> 
>   struct glyph *previous_row__last_glyph
>     = (vpos == 0)
>         ? NULL
>       : (!(row - 1)->enabled_p)
>         ? NULL
>       : (row - 1)->glyphs[TEXT_AREA] + (row - 1)->used[TEXT_AREA] - 1;
> 
> The 'used' member should be positive, or else subtracting 1 is not
> allowed.

Re: Debugging printing to stderr prior glyph row values ....

[Eli Zaretskii]

> Date:  Thu, 23 May 2019 10:23:03 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  emacs-devel@gnu.org
> 
> I will add a test for the `used` row member to ensure that it is positive.  The crashing reported in the initial post only happens somtimes, so I'll need to do some further testing.

Did you look at the problematic row with the GDB command "pgrowx"?
It's important to understand what kind of glyphs are in that row and
how many of them do you have there.

> I am looking to programmatically distinguish between the following three types of consecutive rows:  (1) the penultimate row _not_ ending in ZV that immediately precedes a row with _only_ ZV; (2) the last row that contains _only_ ZV; and, (3) rows that follow a row with _only_ ZV.

There's a 4th case: when a row ends at ZV, but does have glyphs.  This
happens, for example, when visiting a file that doesn't end in a
newline.

> A better method of determining row_at_zv_p (without consulting ROW - 1) would be appreciated, if such a test exists.

I don't understand why you don't just look at the ends_at_zv_p flag,
that's what it's for.

Re: Debugging printing to stderr prior glyph row values ....

[Keith David Bershatsky]

Thank you for mentioning the 4th case; i.e., when a row ends at ZV, but does have glyphs.  Testing for a differential between the start/end row bytepos, coupled with the row->ends_at_zv_p flag being true, appears to be sufficient for that particular case.

The multiple fake cursors feature places fake cursors at particular buffer positions from BEGV to ZV.  The crosshairs feature takes it one step further and draws fake cursors extending beyond ZV until the vertical ruler stops at the top of the mode-line.  As to the multiple fake cursors feature, there is a need to distinguish between a row containing _only_ ZV and subsequent rows so that fake cursors are not drawn there.

The test for row->ends_at_zv_p would be useful for a row containing _only_ ZV (last point in the buffer) if it were the only row with that flag enabled.  As far as I can tell, a row with _only_ ZV (last point in the buffer) is identical to rows that follow it because their ends_at_zv_p flag is true also.  I have not yet been able to figure out a way to programmatically distinguish between 2, 3 and 4 in the following example:

1.  Every good boy deserves fudge.¶
2.  ZV<==
3.  [one or more ' ' space glyphs]
4.  [one or more ' ' space glyphs]
5.  Mode-Line

When the crashing sporadically occurred yesterday, I tried during a couple of crash sessions to call "pgrowx row - 1" and "pgrowx (row - 1)" from gdb, but gdb appeared to produce the output for ROW instead of ROW - 1.  I have now added the following definition in the code so that I can call "pgrowx previous_row" from gdb.

  struct glyph_row *previous_row = row - 1;

I spent several minutes this afternoon trying to make Emacs crash doing things similar to yesterday, but have not yet had any luck getting Emacs to crash ... I will keep trying ...

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [05-23-2019 11:38:25] <23 May 2019 21:38:25 +0300>
> From: Eli Zaretskii <eliz@gnu.org>
> To: Keith David Bershatsky <esq@lawlist.com>
> CC: emacs-devel@gnu.org
> Subject: Re: Debugging printing to stderr prior glyph row values ....
> 
> > Date:  Thu, 23 May 2019 10:23:03 -0700
> > From:  Keith David Bershatsky <esq@lawlist.com>
> > Cc:  emacs-devel@gnu.org
> >
> > I will add a test for the `used` row member to ensure that it is positive.  The crashing reported in the initial post only happens somtimes, so I'll need to do some further testing.
> 
> Did you look at the problematic row with the GDB command "pgrowx"?
> It's important to understand what kind of glyphs are in that row and
> how many of them do you have there.
> 
> > I am looking to programmatically distinguish between the following three types of consecutive rows:  (1) the penultimate row _not_ ending in ZV that immediately precedes a row with _only_ ZV; (2) the last row that contains _only_ ZV; and, (3) rows that follow a row with _only_ ZV.
> 
> There's a 4th case: when a row ends at ZV, but does have glyphs.  This
> happens, for example, when visiting a file that doesn't end in a
> newline.
> 
> > A better method of determining row_at_zv_p (without consulting ROW - 1) would be appreciated, if such a test exists.
> 
> I don't understand why you don't just look at the ends_at_zv_p flag,
> that's what it's for.

Re: Debugging printing to stderr prior glyph row values ....

[Keith David Bershatsky]

I got it to crash ... again.  "pgrowx previous_row" from gdb prints nothing and returns to a new gdb command prompt.

[CAVEAT:  I wanted a crash, so I did not insert a test for the 'used' member being greater than zero.]

Program received signal SIGSEGV, Segmentation fault.
0x0000000100045d3b in mc_draw_row (w=0x11b88b838, matrix=0x11b1d3210,
    row=0x105c89200, start=0x105d24e00, x=0, hpos_length=1, vpos=18,
    bottom_vpos=40, active_p=true, cursor_matrix=0x1011319c0, fc_x=770,
    ch_foreground=..., fc_foreground=..., mc_zv=16, mc_zv_byte=16,
    draw_p=false, from_where=SCRIBE_ONE) at xdisp.c:4605
4605	  fprintf (stderr, "\nA.  vpos (%d) | glyph->charpos (%d) | posint (%d) | zv (%d) | zv_byte (%d)\n\


(gdb) pp w->contents

#<buffer *MC-TEST*>


(gdb) pgrowx previous_row


(gdb) pgrowx row

TEXT: 1 glyphs
  0    0: CHAR[ ] pos=-1 blev=0,btyp=UNDEF w=11 a+d=16+4 MB


(gdb) print vpos

$1 = 18


(gdb) print previous_row

$2 = (struct glyph_row *) 0x105c89100


(gdb) print *previous_row
$3 = {
  glyphs = {0x11a800000, 0x11a800000, 0x11a803520, 0x11a803520},
  used = {0, 0, 0, 0},
  hash = 0,
  x = 0,
  y = 344,
  pixel_width = 11,
  ascent = 16,
  height = 20,
  phys_ascent = 0,
  phys_height = 0,
  visible_height = 20,
  extra_line_spacing = 0,
  start = {
    pos = {
      charpos = 16,
      bytepos = 16
    },
    overlay_string_index = -1,
    string_pos = {
      charpos = -1,
      bytepos = -1
    },
    dpvec_index = -1
  },
  end = {
    pos = {
      charpos = 16,
      bytepos = 16
    },
    overlay_string_index = -1,
    string_pos = {
      charpos = -1,
      bytepos = -1
    },
    dpvec_index = -1
  },
  minpos = {
    charpos = 16,
    bytepos = 16
  },
  maxpos = {
    charpos = 16,
    bytepos = 16
  },
  overlay_arrow_bitmap = 0,
  left_user_fringe_bitmap = 0,
  right_user_fringe_bitmap = 0,
  left_fringe_bitmap = 0,
  right_fringe_bitmap = 0,
  left_user_fringe_face_id = 0,
  right_user_fringe_face_id = 0,
  left_fringe_face_id = 0,
  right_fringe_face_id = 0,
  left_fringe_offset = 0,
  right_fringe_offset = 0,
  fringe_bitmap_periodic_p = false,
  redraw_fringe_bitmaps_p = true,
  enabled_p = true,
  truncated_on_left_p = false,
  truncated_on_right_p = false,
  continued_p = false,
  displays_text_p = false,
  ends_at_zv_p = true,
  fill_line_p = false,
  indicate_empty_line_p = false,
  contains_overlapping_glyphs_p = false,
  full_width_p = false,
  mode_line_p = false,
  overlapped_p = false,
  ends_in_middle_of_char_p = false,
  starts_in_middle_of_char_p = false,
  overlapping_p = false,
  mouse_face_p = false,
  ends_in_newline_from_string_p = false,
  exact_window_width_line_p = false,
  cursor_in_fringe_p = false,
  ends_in_ellipsis_p = false,
  indicate_bob_p = false,
  indicate_top_line_p = false,
  indicate_eob_p = false,
  indicate_bottom_line_p = false,
  reversed_p = false,
  continuation_lines_width = 0,
  clip = 0x0
}

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [05-23-2019 11:38:25] <23 May 2019 21:38:25 +0300>
> From: Eli Zaretskii <eliz@gnu.org>
> To: Keith David Bershatsky <esq@lawlist.com>
> CC: emacs-devel@gnu.org
> Subject: Re: Debugging printing to stderr prior glyph row values ....
>
> > Date:  Thu, 23 May 2019 10:23:03 -0700
> > From:  Keith David Bershatsky <esq@lawlist.com>
> > Cc:  emacs-devel@gnu.org
>
> > I will add a test for the `used` row member to ensure that it is positive.  The crashing reported in the initial post only happens somtimes, so I'll need to do some further testing.
>
> Did you look at the problematic row with the GDB command "pgrowx"?
> It's important to understand what kind of glyphs are in that row and
> how many of them do you have there.
>
> * * *

Re: Debugging printing to stderr prior glyph row values ....

[Eli Zaretskii]

> Date:  Thu, 23 May 2019 17:03:36 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  emacs-devel@gnu.org
> 
> I got it to crash ... again.  "pgrowx previous_row" from gdb prints nothing and returns to a new gdb command prompt.

This means the previous glyph row had its 'used' member equal to zero.

> [CAVEAT:  I wanted a crash, so I did not insert a test for the 'used' member being greater than zero.]

And it shows:

> (gdb) print *previous_row
> $3 = {
>   glyphs = {0x11a800000, 0x11a800000, 0x11a803520, 0x11a803520},
>   used = {0, 0, 0, 0},

See? all of the 'used' members are zero.  IOW, there are no glyphs in
that row.

Re: Debugging printing to stderr prior glyph row values ....

[Keith David Bershatsky]

Thank you, Eli, for teaching me how to debug this particular issue and also how to avoid crashes of this nature in the future.  I was unaware that an enabled_p (true) row could have zero glyphs.  Now that I know this is possible, I can programmatically deal with it.

Inasmuch as rows following a row with _only_ ZV contain an ends_at_zv_p flag that is also true, and because there may be no guarantee that the preceding row in the matrix can be relied upon to ascertain whether it ends at ZV, I believe a new flag would be helpful (at least in this particular context) -- i.e., beyond_zv_p versus ends_at_zv_p.  That way, only one row in the entire matrix will have a flag that indicates an at-ZV versus a post-ZV situation.  I'll spend some time on Saturday digging around xdisp.c ...

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [05-23-2019 23:45:44] <24 May 2019 09:45:44 +0300>
> From: Eli Zaretskii <eliz@gnu.org>
> To: Keith David Bershatsky <esq@lawlist.com>
> CC: emacs-devel@gnu.org
> Subject: Re: Debugging printing to stderr prior glyph row values ....
> 
> > Date:  Thu, 23 May 2019 17:03:36 -0700
> > From:  Keith David Bershatsky <esq@lawlist.com>
> > Cc:  emacs-devel@gnu.org
> >
> > I got it to crash ... again.  "pgrowx previous_row" from gdb prints nothing and returns to a new gdb command prompt.
> 
> This means the previous glyph row had its 'used' member equal to zero.
> 
> > [CAVEAT:  I wanted a crash, so I did not insert a test for the 'used' member being greater than zero.]
> 
> And it shows:
> 
> > (gdb) print *previous_row
> > $3 = {
> >   glyphs = {0x11a800000, 0x11a800000, 0x11a803520, 0x11a803520},
> >   used = {0, 0, 0, 0},
> 
> See? all of the 'used' members are zero.  IOW, there are no glyphs in
> that row.