bug#8627: 24.0.50: cursor property behaves irregularly in before-strings

[Alp Aker <aker@pitt.edu>]

>> But if the before-string contains a newline, the cursor property
>> appears to be ignored.
>
> I will take a look at fixing this, but can you show a real-life use
> case where this is needed?

I ran into the need while working on this package:

   emacswiki.org/emacs/FillColumnIndicator

It shades the area past the fill column by putting overlays on newlines 
(if you look at the screenshot it'll be obvious how it works).  In a few 
edge cases it would be useful to be able to display a newline that's not 
part of the buffer content.  For example, if the last line of the buffer 
doesn't end in a newline, then no fill-column shading appears on that line 
(since there's no newline to propertize).  I'd like to put an overlay at 
point-max with an after-string containing a propertized newline character, 
so that the fill-column shading can appear on the last line of the buffer. 
But I can't do so, because the newline in the after-string causes any 
cursor property in it to be ignored, and so cursor motion at the end of 
the buffer becomes deranged.  (Display strings won't do here, because I 
don't want to replace the display of any characters in the buffer.)

Does that qualify as a use case?

I might as well also mention, at least for the record, another (minor) 
issue with cursor properties in before-strings that I came across:  If the 
position with the cursor property is out of sight because of horizontal 
scrolling, then the property is ignored, even when auto-hscroll-mode is 
enabled.  Perhaps it would make more sense to respect the cursor property 
when auto-hscroll-mode is non-nil, and adjust hscrolling to bring the 
requested cursor position into view?

Here's a test case:

(progn
   (setq auto-hscroll-mode t)
   ;; A line of text, moving to the end of which will trigger hscrolling.
   (insert (make-string (+ hscroll-margin (window-width) 5) ?X))
   (setq m (point-marker))
   (insert "\n\n")
   (setq o (make-overlay (1- (point)) (point)))
   ;; Request that cursor be at bol, but make the before-string long
   ;; enough that if the cursor property is ignored we won't return from
   ;; hscrolling.
   (overlay-put o
                'before-string
                (concat
                 (propertize " " 'cursor 1)
                 (make-string (window-width) 32)))
   (goto-char m)
   (redisplay)
   (forward-line 1))

After evaluation the cursor will be at the end of the before-string, 
rather than at the requested position.  (This happens with 22 and 23 as 
well.)