How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

[Keith David Bershatsky]

I am looking for some assistance, please, to obtain in C (e.g., xdisp.c) the values for x/y/hpos/vpos at the end of a visual line when overlay 'after-string exists.

Whenever there is an overlay 'after-string at the end of the line, the `x` and `hpos` coordinates for eol are not what I want -- i.e., the values for `x` and `hpos` are both increased depending upon the visual length of the 'after-string.

I have tried adding `it->ignore_overlay_strings_at_pos_p = true;` to custom functions similar to `move_it_to` and `move_it_in_display_line_to`.  However, that doesn't seem to have the desired affect.

I have also tried deleting all overlays at the beginning of the command loop, but the overlays seem to still be present until redisplay finishes.

Here is what I have been using:

  struct it it;
  struct text_pos pt;
  /* peovl is shorthand for `point at end of visual line`.  */
  EMACS_INT peovl, peovl_x, peovl_y, peovl_hpos, peovl_vpos;
  peovl = [let us assume this is a point at the end of a visual line subsequent to w->start, either wrapped or just a plain end of line];
  SET_TEXT_POS_FROM_MARKER (pt, w->start);
  start_display (&it, w, pt);
  move_it_to (&it, peovl, -1, -1, -1, MOVE_TO_POS);
  peovl = CHARPOS (it.current.pos);
  peovl_x = it.current_x;
  peovl_y = it.current_y;
  peovl_hpos = it.hpos;
  peovl_vpos = it.vpos;


Thanks,

Keith

Re: How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

[Eli Zaretskii]

> Date: Mon, 04 Apr 2016 12:39:18 -0700
> From: Keith David Bershatsky <esq@lawlist.com>
> 
> I am looking for some assistance, please, to obtain in C (e.g., xdisp.c) the values for x/y/hpos/vpos at the end of a visual line when overlay 'after-string exists.
> 
> Whenever there is an overlay 'after-string at the end of the line, the `x` and `hpos` coordinates for eol are not what I want -- i.e., the values for `x` and `hpos` are both increased depending upon the visual length of the 'after-string.

Please tell what problem you are trying to solve, and why.  Obtaining
coordinates is a means, but I don't yet see the end, so please help us
understand that.

Re: How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

[Keith David Bershatsky]

[-- Attachment #1: Type: text/plain, Size: 134 bytes --]

I forgot to cc the mailing list -- so here is a carbon copy.

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;


[-- Attachment #2: Type: message/rfc822, Size: 2765 bytes --]

From: Keith David Bershatsky <esq@lawlist.com>
To: Eli Zaretskii <eliz@gnu.org>
Subject: Re: How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.
Date: Mon, 04 Apr 2016 14:06:08 -0700
Message-ID: <m2oa9paxtr.wl%esq@lawlist.com>

Thank you, Eli, for taking a look at this particular thread.

There are a couple of applications where this would be useful.

The first application is a common function available interactively called `posn-at-point` which cannot return the desired `x` and `hpos` when an 'after-string is present at point.  For example, line-move-visual in simple.el has never worked correctly when I have overlay 'after-string at the end of the line.  So, I have been using a workaround with `vertical-motion` to calculate the desired `temporary-goal-column`.  Here is the link to a thread on emacs.stackexchange.com from about a year ago relating to this issue:

http://emacs.stackexchange.com/questions/7957/calculating-cursor-position-excluding-the-overlay-after-string

The second application is related to feature request 22873 (multiple fake cursors) and a dormant (yet related) feature request 17684 (drawing a vertical line the length of the window).  I have not been able to accurately predict programmatically where the fake cursors are located subsequent to the screen scrolling when overlay 'after-string are present.  [I spent a day writing up a function to calculate the differential of `y` and `vpos` based on previous and current window-start/end, only to discover that the Little Rascals (fake cursors) were still somewhere else when scrolling.]  The function `erase_phys_cursor` redraws the character (where the cursor was) based upon (in part) the values of x/y/hpos/vpos.  Until I can figure out how to accurately predict where those Little Rascals (fake
  cursors) have gone to when scrolling occurs, the easiest thing to do is erase/redraw everything along the `x` axis.  `erase_phys_cursor` should not be used, however, when there is no TEXT along the `x` axis - because that yields strange looking glyphs along the right side of the screen.  Before calling `erase_phys_cursor` (or my custom function `mc_erase`), I want to check to see where the end of the visual line is -- i.e., x/y/hpos/vpos at the end of each visual line -- and only erase/redraw characters when the `x` axis is less than the end of the visual line

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At Mon, 04 Apr 2016 23:30:04 +0300,
Eli Zaretskii wrote:
> 
> * * *
> 
> Please tell what problem you are trying to solve, and why.  Obtaining
> coordinates is a means, but I don't yet see the end, so please help us
> understand that.

Re: How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

[Eli Zaretskii]

> Date:  Mon, 04 Apr 2016 14:06:08 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> 
> There are a couple of applications where this would be useful.
> 
> The first application is a common function available interactively called `posn-at-point` which cannot return the desired `x` and `hpos` when an 'after-string is present at point.  For example, line-move-visual in simple.el has never worked correctly when I have overlay 'after-string at the end of the line.  So, I have been using a workaround with `vertical-motion` to calculate the desired `temporary-goal-column`.  Here is the link to a thread on emacs.stackexchange.com from about a year ago relating to this issue:
> 
> http://emacs.stackexchange.com/questions/7957/calculating-cursor-position-excluding-the-overlay-after-string

Your original request was about doing this on the C level, so the
above use case is not really related: line-move-visual is implemented
in Lisp.

> The second application is related to feature request 22873 (multiple fake cursors) and a dormant (yet related) feature request 17684 (drawing a vertical line the length of the window).  I have not been able to accurately predict programmatically where the fake cursors are located subsequent to the screen scrolling when overlay 'after-string are present.  [I spent a day writing up a function to calculate the differential of `y` and `vpos` based on previous and current window-start/end, only to discover that the Little Rascals (fake cursors) were still somewhere else when scrolling.]  The function `erase_phys_cursor` redraws the character (where the cursor was) based upon (in part) the values of x/y/hpos/vpos.  Until I can figure out how to accurately predict where those Little Rascals (fa
 ke cursors) have gone to when scrolling occurs, the easiest thing to do is erase/redraw everything along the `x` axis.  `erase_phys_cursor` should not be used, however, when there is no TEXT along the `x` axis - because that yields strange looking glyphs along the right side of the screen.  Before calling `erase_phys_cursor` (or my custom function `mc_erase`), I want to check to see where the end of the visual line is -- i.e., x/y/hpos/vpos at the end of each visual line -- and only erase/redraw characters when the `x` axis is less than the end of the visual line

This is indeed about something you do in C.  However, I don't
understand why you try using posn-at-point, which is a Lisp API, for a
job you need to do in C.  On the C level, you have access to a more
elaborate information, e.g. you can examine the object at point and do
something when it is an overlay string.

I must admit I still don't see clearly why you need screen coordinates
corresponding to a buffer position, in the multiple-curses scenario.
Isn't it your code that positions those cursors in the first place?
If so, why do you need to find out where those cursors are, if your
code did the positioning?

Re: How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

[Keith David Bershatsky]

There two (2) major roadblocks that I have encountered with multiple cursors:

PROBLEM # 1:  `mc_x_y_hpos_vpos` needs to be modified to exclude overlays from the calculation so that placement and removal of fake cursors can be correctly achieved.  Any assistance in modifying this function to exclude overlays from the calculation would be greatly appreciated.

    /* The mission critical function used by multiple cursors to extract coordinates.  */
    Lisp_Object
    mc_x_y_hpos_vpos (struct window *w, EMACS_INT posint, EMACS_INT start, EMACS_INT end)
    {
      struct it it;
      void *itdata = bidi_shelve_cache ();
      struct text_pos pt, pos;
      int x, y, hpos, vpos;
      if (posint >= start
          && posint <= end)
        {
          SET_TEXT_POS_FROM_MARKER (pt, w->start);
          start_display (&it, w, pt);
          move_it_to (&it, posint, -1, -1, -1, MOVE_TO_POS);
          pos = it.current.pos;
          x = it.current_x;
          y = it.current_y;
          hpos = it.hpos;
          vpos = it.vpos;
          bidi_unshelve_cache (itdata, false);
        }
        else
          {
            x = -1;
            y = -1;
            hpos = -1;
            vpos = -1;
          }
      return listn (CONSTYPE_PURE, 4, make_number (x), make_number (y), make_number (hpos), make_number (vpos));
    }


PROBLEM # 2:  When no scrolling occurs, the cache of previously laid multiple cursors is sufficient to remove them.  When scrolling occurs (less than a full screen) -- e.g., previous/next-line or scroll-up/down -- the multiple cursors become moving targets.  In that scenario, the previously recorded x/y/hpos/vpos will not suffice.  I tried something elaborate to calculate the differential of `y` and `vpos` based on scrolling; however, that didn't work.  Either my calculations are way off, or I do not understand what is happening when scrolling occurs.  My idea was that I could add or subtract the differential from the stored values in the cache to pinpoint the new locations for the previously laid multiple cursors whenever scrolling occurred.

LEGEND:
* `start`:  window-start.
* `end`:  window-end.
* `prev`:  value from the prior command loop.
* `opoint`:  original point.
* `y_diff`:  differential of `y` coordinate between command loops.
* `vpos_diff:  differential of `vpos` between command loops.

  if ((!NILP (BVAR (current_buffer, mc_list)) || w->mc_cache)
      && !MINI_WINDOW_P (w))
    {
      Lisp_Object x_y_hpos_vpos_list;
      EMACS_INT start = CHARPOS (startp);
      EMACS_INT end = BUF_Z (buffer) - w->window_end_pos;
      if (w->mc.set_start_end)
        {
          if (w->mc.start)
            {
              w->mc.prev_start = w->mc.start;
              x_y_hpos_vpos_list = mc_x_y_hpos_vpos (w, w->mc.prev_start, start, end);
              w->mc.prev_start_x = XINT (Fnth (make_number (0), x_y_hpos_vpos_list));
              w->mc.prev_start_y = XINT (Fnth (make_number (1), x_y_hpos_vpos_list));
              w->mc.prev_start_hpos = XINT (Fnth (make_number (2), x_y_hpos_vpos_list));
              w->mc.prev_start_vpos = XINT (Fnth (make_number (3), x_y_hpos_vpos_list));
            }
            else
              {
                w->mc.prev_start = -1;
                w->mc.prev_start_x = -1;
                w->mc.prev_start_y = -1;
                w->mc.prev_start_hpos = -1;
                w->mc.prev_start_vpos = -1;
              }
          if (w->mc.end)
            {
              w->mc.prev_end = w->mc.end;
              /* eob IS visible].
              Excludes a situation when last visible line at bottom of window contains eob.  */
              if (w->mc.prev_end == end)
                {
                  x_y_hpos_vpos_list = mc_x_y_hpos_vpos (w, w->mc.prev_end, start, end);
                  w->mc.prev_end_x = w->mc.end_x;
                  w->mc.prev_end_y = w->mc.end_y;
                  w->mc.prev_end_hpos = w->mc.end_hpos;
                  w->mc.prev_end_vpos = w->mc.end_vpos;
                }
                /* eob is NOT visible.
                Excludes a situation when last visible line at bottom of window contains eob.  **/
                else
                  {
                    x_y_hpos_vpos_list = mc_x_y_hpos_vpos (w, w->mc.prev_end, start, end);
                    w->mc.prev_end_x = XINT (Fnth (make_number (0), x_y_hpos_vpos_list));
                    w->mc.prev_end_y = XINT (Fnth (make_number (1), x_y_hpos_vpos_list));
                    w->mc.prev_end_hpos = XINT (Fnth (make_number (2), x_y_hpos_vpos_list));
                    w->mc.prev_end_vpos = XINT (Fnth (make_number (3), x_y_hpos_vpos_list));
                  }
            }
            else
              {
                w->mc.prev_end = -1;
                w->mc.prev_end_x = -1;
                w->mc.prev_end_y = -1;
                w->mc.prev_end_hpos = -1;
                w->mc.prev_end_vpos = -1;
              }
          if (w->mc.opoint)
            {
              w->mc.prev_opoint = w->mc.opoint;
              w->mc.prev_opoint_x = w->mc.opoint_x;
              w->mc.prev_opoint_y = w->mc.opoint_y;
              w->mc.prev_opoint_hpos = w->mc.opoint_hpos;
              w->mc.prev_opoint_vpos = w->mc.opoint_vpos;
            }

          /* FIXME -- the X and HPOS will always be wrong if point is at an overlay after-string.  */
          w->mc.opoint = PT;
          x_y_hpos_vpos_list = mc_x_y_hpos_vpos (w, w->mc.opoint, start, end);
          w->mc.opoint_x = XINT (Fnth (make_number (0), x_y_hpos_vpos_list));
          w->mc.opoint_y = XINT (Fnth (make_number (1), x_y_hpos_vpos_list));
          w->mc.opoint_hpos = XINT (Fnth (make_number (2), x_y_hpos_vpos_list));
          w->mc.opoint_vpos = XINT (Fnth (make_number (3), x_y_hpos_vpos_list));

          w->mc.start = start;
          x_y_hpos_vpos_list = mc_x_y_hpos_vpos (w, start, start, end);
          w->mc.start_x = XINT (Fnth (make_number (0), x_y_hpos_vpos_list));
          w->mc.start_y = XINT (Fnth (make_number (1), x_y_hpos_vpos_list));
          w->mc.start_hpos = XINT (Fnth (make_number (2), x_y_hpos_vpos_list));
          w->mc.start_vpos = XINT (Fnth (make_number (3), x_y_hpos_vpos_list));

          w->mc.end = end;
          x_y_hpos_vpos_list = mc_x_y_hpos_vpos (w, end, start, end);
          w->mc.end_x = XINT (Fnth (make_number (0), x_y_hpos_vpos_list));
          w->mc.end_y = XINT (Fnth (make_number (1), x_y_hpos_vpos_list));
          w->mc.end_hpos = XINT (Fnth (make_number (2), x_y_hpos_vpos_list));
          w->mc.end_vpos = XINT (Fnth (make_number (3), x_y_hpos_vpos_list));

          /* Start remains the same.  */
          if (w->mc.prev_start_vpos == w->mc.start_vpos
              || w->mc.prev_end < w->mc.start)
            {
              w->mc.y_diff = 0;
              w->mc.vpos_diff = 0;
            }
            /* Screen moved ▼ -- e.g., `previous-line', `scroll-down'.  */
            else if (w->mc.prev_start_vpos > 0)
              {
                w->mc.y_diff = (w->mc.prev_start_y - w->mc.start_y) * -1;
                w->mc.vpos_diff = (w->mc.prev_start_vpos - w->mc.start_vpos) * -1;
              }
              /* Screen moved ▲ (eob NOT visible) -- e.g., `next-line', `scroll-up'.  */
              else if (w->mc.end_vpos > w->mc.prev_end_vpos && w->mc.prev_end != -1)
                {
                  w->mc.y_diff = w->mc.end_y - w->mc.prev_end_y;
                  w->mc.vpos_diff = w->mc.end_vpos - w->mc.prev_end_vpos;
                }
                /* Screen moved ▲ (eob IS visible) -- e.g., `next-line', `scroll-up'.  */
                else if (w->mc.end_vpos < w->mc.prev_end_vpos && w->mc.prev_end != -1)
                  {
                    w->mc.y_diff = w->mc.prev_end_y - w->mc.end_y;
                    w->mc.vpos_diff = w->mc.prev_end_vpos - w->mc.end_vpos;
                  }
                  /* Screen moved ▲ (eob just became visible) -- e.g., `next-line', `scroll-up'.
                  Absent subtracting 1 from w->mc.prev_end, that point and the new w->mc.end are
                  on the same line and it appears that nothing has changed -- w->mc.prev_end is
                  on the far left of the line and w->mc.end is on the far right of the line.  */
                  else if (w->mc.end_vpos == w->mc.prev_end_vpos)
                    {
                      x_y_hpos_vpos_list = mc_x_y_hpos_vpos (w, w->mc.prev_end - 1, start, end);
                      w->mc.prev_end_x = XINT (Fnth (make_number (0), x_y_hpos_vpos_list));
                      w->mc.prev_end_y = XINT (Fnth (make_number (1), x_y_hpos_vpos_list));
                      w->mc.prev_end_hpos = XINT (Fnth (make_number (2), x_y_hpos_vpos_list));
                      w->mc.prev_end_vpos = XINT (Fnth (make_number (3), x_y_hpos_vpos_list));
                      w->mc.y_diff = w->mc.end_y - w->mc.prev_end_y;
                      w->mc.vpos_diff = w->mc.end_vpos - w->mc.prev_end_vpos;
                    }
                /* `mc-list` has just been initialized -- previous start/end are set to -1.  */
                    else if (w->mc.prev_end == -1)
                      {
                        w->mc.y_diff = 0;
                        w->mc.vpos_diff = 0;
                      }

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At Tue, 05 Apr 2016 18:08:09 +0300,
Eli Zaretskii wrote:
> 
> * * *
> 
> This is indeed about something you do in C.  However, I don't
> understand why you try using posn-at-point, which is a Lisp API, for a
> job you need to do in C.  On the C level, you have access to a more
> elaborate information, e.g. you can examine the object at point and do
> something when it is an overlay string.
> 
> I must admit I still don't see clearly why you need screen coordinates
> corresponding to a buffer position, in the multiple-curses scenario.
> Isn't it your code that positions those cursors in the first place?
> If so, why do you need to find out where those cursors are, if your
> code did the positioning?

Re: How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

[Eli Zaretskii]

> Date:  Tue, 05 Apr 2016 10:37:44 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  emacs-devel@gnu.org
> 
> PROBLEM # 1:  `mc_x_y_hpos_vpos` needs to be modified to exclude overlays from the calculation so that placement and removal of fake cursors can be correctly achieved.  Any assistance in modifying this function to exclude overlays from the calculation would be greatly appreciated.

Actually, overlay strings is just one possible way of having a buffer
position concealed in the glyph row's glyphs.  There are a few more,
like display strings, selective display, invisible text, etc.  A
display or overlay string can also have a 'cursor' property on one of
its characters, and users will expect you to place the cursor on that
character.  All of those situations are problematic for finding the
cursor coordinates given just its buffer position.  For the full
story, see set_cursor_from_row.

So I think your design is sub-optimal.  AFAIU, you wait until
redisplay is done with its job and calls the frame's
update_window_end_hook.  Then you do all of the job of positioning the
additional cursors and drawing them in that hook.  This is suboptimal,
because you are forced to traverse the window anew starting from its
beginning for each additional cursor.  Instead, you could compute the
cursor coordinates as part of the layout code, like what we do with
set_cursor_from_row.  The results of that function, i.e. the
coordinates of the cursor in the window, are stored in the window
object, and update_window_end_hook just uses them to actually draw the
cursor.  You could refactor set_cursor_from_row and the code that
calls it to compute the coordinates of additional cursors, and then
you'd get the solution to your problems with overlays and whatnot for
free, because set_cursor_from_row already deals with all that.  And
your 2nd problem will go away as well, because the cursor coordinates
will be computed as part of the code which scrolls the display,
instead of trying to find the previous cursor position after scrolling
was already done.

You should also look at expose_frame and its subroutines, because
handling the expose events requires redrawing the cursors, which AFAIU
you currently don't do.

Re: How to obtain x/y/hpos/vpos at eol -- excluding overlay 'after-string.

[Keith David Bershatsky]

Thank you, Eli, for the outline/blueprint of the best method for tackling the issue of multiple cursors.

As always, your tutoring is greatly appreciated.

I will continue working on this project in my spare time.

Sincerely,

Keith