bug#11094: Wrong cursor positioning with display+invisible

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> Extract the test.cpio and cpio-mode.el files attached, and then try:
>> src/emacs -Q -l .../cpio-mode.el .../test.cpio
> Ouch!

;-)

> (Btw, test.cpio is an invalid archive, according to 2 different
> implementations of the cpio command.  The one from GNU/Linux says
> "cpio: premature end of file".  Not that it matters for the purposes
> of this discussion.)

I just truncated it, since the original (an initrd) is 10MB long.

>> In Emacs-23, the cursor jumps correctly over the various display and
>> invisible fields
> For some value of "correctly".  E.g., position the cursor over the
> ".", which is the first file name in the archive, and type "C-x =".
> You will see "63", which is a lie: the actual value of point is 111.

I don't think it's a lie because that text is preceded by invisible
text, so while "." is at 111, point is indeed at 63 (and that's just
because of adjust_point_for_property, so we can get point to stay at
111 by being more careful with text property's stickiness).

> The old, unidirectional display engine could get away (albeit by the
> skin of its teeth) with such code because it relied on buffer
> positions to increase monotonically with screen positions.  So once it
> found a glyph from buffer position N that is greater or equal to the
> value of point, it could place the cursor there, because it "knew" all
> the later glyphs _must_ correspond to positions beyond N.  That is why
> it "works" in Emacs 23.

I see that the problem is not so much that all the text is covered by
those properties, but rather that there is are contiguous texts with
`invisible' and `display' properties.  I get Emacs-24 to behave
correctly by turning all "invisible span followed by display span" into
just a single display span.

E.g. if you do

  emacs -Q
  (put-text-property (point-min) (+ 2 (point-min)) 'invisible t)
  (put-text-property (+ 2 (point-min)) (+ 4 (point-min)) 'display "<>")
  (goto-char (point-min))

you'll see that the cursor is drawn at the wrong place (i.e. after the
<>), whereas if you do
  
  emacs -Q
  (put-text-property (point-min) (+ 4 (point-min)) 'display "<>")
  (goto-char (point-min))

the cursor is drawn at the right place.  I think this is the core of the
problem that's handled differently from Emacs-23.
[ IIUC you've gotten to the same conclusion.  ]

> If you still think we should support your original code, we should
> schedule some post-24.1 redesign and refactoring.  Let me know what
> you think.

I don't think it's a very high priority problem, but it would be good to
try and tackle it, yes (post-24.1).


        Stefan