bug#27427: 26.0.50; Native line numbers lead to display error in company-mode popup

[Eli Zaretskii <eliz@gnu.org>]

> Date: Wed, 28 Jun 2017 10:45:26 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, alexanderm@web.de, 27427@debbugs.gnu.org
> 
>  > But the display engine is not involved in the TTY menus, at least not
>  > in the usual sense.  Please take a look at the second half of
>  > display_menu_bar, and you will see what I mean.
> 
> Isn't display_menu_bar the code for displaying a non-X GUI menu?  I
> suppose you allude to the "while (!leave)" part of tty_menu_activate.
> Please correct me if I'm wrong.  I don't understand why this should be
> relevant for popup frames, probably because I'm too silly.

Sorry, you are right.  I meant display_tty_menu_item.

> An overlay with a 'tty-popup' property would specify a buffer position
> and a newline separated text.  The buffer position would implicitly
> provide the upper left corner of the popup frame, the text its size.
> 
> The display engine would notice the overlay like any other overlay.
> However, the 'tty-popup' property would cause it to (1) remember the
> current column of the iterator and (2) draw the first line of the
> overlay right where the iterator is at this moment, possibly clipping
> this line of the overlay text at the frame edge and skipping as many
> characters of underlying buffer text as there are on this overlay text
> line.
> 
> On the next buffer text line the display engine would start to draw the
> second line of the overlay text at the column remembered in (1),
> clipping and skipping as on the first line.  It would continue to
> process the overlay until either its text has been used up or the bottom
> of the window or the frame's root window has been reached.

Alas, that is not how redisplay works.  For starters, you seem to
assume that it always traverses all the screen lines of a window (thus
"on the next buffer line" etc.).  But that is only so when a window
needs a complete redisplay, and the display engine tries very hard to
avoid that.  Many times it only redraws some of the lines, sometimes
just one line.  If that line is not the first line of the popup, how
will the display engine know that at some previous buffer position
there is an overlay?

Moreover, the same low-level display code is invoked by the move_it_*
functions which simulate display without drawing anything, and those
are very often invoked to traverse small portions of a buffer,
typically a small number of lines.  These will be in trouble as well,
for the same reason.

So we need a different solution, one that doesn't break due to
redisplay optimizations.

Dmitry, can you tell why the popup overlay is a single overlay with a
single multiline string, and not a series of overlays, one each for
every line shown in the popup?  I assume this caused or could cause
more serious problems than the current implementation, but what
problems were those?