bug#8627: 24.0.50: cursor property behaves irregularly in before-strings

[Eli Zaretskii <eliz@gnu.org>]

> Date: Thu, 05 May 2011 20:01:00 -0400 (EDT)
> From: Alp Aker <aker@pitt.edu>
> 
> However, when used with overlay before-strings (or after-strings), the 
> cursor property appears to behave in ways that aren't consonant with the 
> docs and that don't follow a consistent pattern.

I took a quick look at this bug.  I must say I wish this feature were
never introduced into Emacs: it is hackyish, its implementation is
kludgey, and supporting it in the bidirectional display engine will be
a PITA.  It also makes no sense at all in some use cases.  E.g., if
you put a `cursor' property with an integer value of 1 on either a
before-string or an after-string, the cursor will never be positioned
on the character at point, although it is clearly visible.  And if the
value is greater than 1, the cursor will appear to be "stuck" for that
many C-f keypresses.  I'm sure some users will think it's a bug.

Anyway, I will eventually fix the code to do what it did in Emacs 23,
with one exception:

> (3) But if the before-string contains a newline, the cursor property 
> appears to be ignored regardless of location and regardless of whether the 
> value of the cursor property is numeric or merely non-nil.  With the 
> overlay still in the middle of the line:
> 
>    (setq str3 (concat str1 "\nWWW")
>          str4 (concat str2 "\nWWW"))
>    (overlay-put olay 'before-string str3)
>    (goto-char (overlay-start olay))
>    (overlay-put olay 'before-string str4)
>    (goto-char (overlay-start olay))
> 
> the cursor appears after the before string, for both types of property 
> value.

This cannot possibly work, not without rewriting the Emacs display
engine in ways I don't intend to.  Quite simply, you cannot put the
`cursor' property on a newline that belongs to a string, because a
newline, obviously, doesn't have a graphic representation (a glyph) on
the screen, it just causes Emacs to continue drawing on the next
screen line.

The function that decides where to position the cursor scans the
glyphs on the line that contains point, looking for a glyph that came
from some string position that has a non-nil `cursor' property.  If it
finds such a glyph, it then does what you expect.  But it will never
find a glyph that corresponds to a newline, and so the `cursor'
property on a newline that comes from a before- or after-string will
never be noticed.  It is effectively lost in the data structure
(called a `glyph matrix') which Emacs uses to draw and redraw the
screen.

In fact, with a newline from a string, Emacs doesn't even consider the
screen line where you wanted it to put the cursor be a candidate for
drawing the cursor, because the newline moves the buffer position
"covered" by the overlay to the next screen line.

I will eventually update the documentation with this caveat.

It's too bad that this is exactly the feature that you needed, but you
will have to rethink how to get the same effect.  I suggest putting
the overlay on some different character in the string, and giving the
`cursor' property a value > 1, maybe that will do what you want
(although I admit that I don't quite understand your exact situation).

> In any case, it would appear that either the treatment of before-strings 
> and after-strings should be changed so that the cursor property works with 
> them as it does with strings that come from display properties

That's not possible, because there's a fundamental difference between
the `display' properties whose values are strings and before- and
after-string properties: the former are _replacing_ properties,
i.e. these strings are displayed _instead_ of the buffer text covered
by the property/overlay.  By contrast, before- and after-strings do
not replace the text that is covered by the overlay.  The code that
treats these two use cases is thus different in pretty much
fundamental ways.