bug#20847: [display engine] 25.0.50; company-mode popup makes point jump to an entirely different location

[Eli Zaretskii <eliz@gnu.org>]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 19 Jun 2015 04:04:45 +0300
> 
> 1. Install company (from GNU ELPA, for instance).
> 
> 2. Paste the contents of
> https://gist.githubusercontent.com/sooheon/97a62f433897b52da3d1/raw/a42af658ec3ccd11a4faa7e2581f7413687b1811/gistfile1.txt
> into the scratch buffer.
> 
> 3. Look for the line with "hello halleo helo" at the end of the first
> paragraph, and delete or add a few characters before it, to make sure
> that the last "h" is two columns away from the right window border. So
> as when you type "el" after it, the cursor is displayed in the margin.
> 
> 4. M-x company-mode
> 
> 5. (setq company-backends '(company-dabbrev))
> 
> 6. Go to after the last "h", type "el", wait 0.3 seconds, see the
> completion popup displayed, while the cursor is displayed in the margin.
> 
> Problem 1: even though the overlay's `cursor' property places it at the
> same line as where "hel" ends, the cursor is rendered in the margin two
> lines below.

I'm halfway through investigating this, and this is what I saw till
now:

 . There's no 'cursor' property on the overlay string that Company
   creates for its "tooltip" of completions.  Or at least I couldn't
   find that property: I tried both "M-x describe-text-properties" and
   looking at the overlay string in GDB -- I see no 'cursor' property,
   certainly not on the newline that starts the string.  If indeed
   there is such a property there, please tell how to see it.

 . In any case, you cannot put the 'cursor' property on a newline and
   hope it to work: the newline doesn't leave any glyphs on display,
   certainly not when the cursor is displayed on the fringe.  So the
   display engine doesn't know you've put the 'cursor' property there.

 . The overlay string generated by Company in this case is
   problematic: it puts a newline at the end of each screen line, and
   that removes the last character of each screen line from display.
   I don't understand why you need to insert newlines when the
   original text lines were one long continued line.  This is a bug in
   Company.

> Problem 2:
> 
> 7. Type "l", see it wrapped to the next line.
> 
> 8. Type backspace. See the cursor move to the second paragraph.
> 
> 9. Continue backspacing. See the completion popup disappear, and the text
> being deleted in the second paragraph.

I didn't yet finish debugging this part, but I clearly see that some
code actually _moves_ point to that place in the second paragraph, I'm
not yet sure why.  For starters, if you turn off font-lock in the
buffer, this second problem doesn't happen at all.  I see some weird
interaction between JIT Font Lock and the post-command-hook installed
by Company, they seem to somehow conspire to force point to move to
that place.  I'll try to debug more to see why this happens.  (Any
idea why Company's post-command-hook calls sit-for, thus forcing
redisplay?)