bug#17244: 24.3.90; `line-move-visual' errors when moving across wrapped lines with an overlay property of 'display

[Ivan Andrus <darthandrus@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2948 bytes --]

On Apr 14, 2014, at 2:11 AM, Eli Zaretskii <eliz@gnu.org> wrote:

From: Ivan Andrus <darthandrus@gmail.com>
Date: Sun, 13 Apr 2014 19:43:08 -0600
Cc: 17244-done@debbugs.gnu.org

<rant>
To tell the truth, Lisp code which covers large portions of buffer
text with much shorter display strings that include newlines deserves
to be broken.  The current Emacs display engine was never designed to
handle situations where the displayed text is so starkly different
from buffer text, so the result of trying to fix "bugs" such as this
one is a never-ending series of band-aids, one upon the other, which
make the code utterly incomprehensible and unmaintainable.

So I'm this close to refusing to fix such "bugs", and instead asking
the authors of such Lisp to either find more benign ways of expressing
what they need, or work around the limitations of the display engine
in their own Lisp.
</rant>


So is the problem having newlines in the 'display property, or hiding large
portions of the buffer?


Each one of these is a problem.  When they are present together in the
same display string, the problems grow exponentially.

Because, in my original use case (my modified version of
http://www.emacswiki.org/emacs/fold.el) I don’t think I have newlines in
the display property, though I definitely hide large portions of the buffer.


This is enough to cause problems.  Consider what Emacs needs to do
when you invoke "C-u N C-p" or "(line-move-visual -N)".  It needs to
find out what is the buffer position N screen lines before point.  But
where in the buffer is that?  When those N lines display what is
mostly buffer text, we can go back the appropriate amount of physical
lines, and then start looking from there.  But if what's on display in
those N lines has little if any resemblance to what's in the buffer,
the only safe algorithm is to go to the beginning of the buffer and
start from there, which is prohibitively expensive.  So Emacs
implements several optimizations to make the operations reasonably
fast.  These optimizations only succeed when the display strings and
the text they cover are approximately similar in length, or if the
display strings are very short.  The specific optimization that caused
this bug was an attempt to be faster when text lines in a buffer are
very long.


I see.  Thanks for explaining.

The overlays are also nested, so it might it be related to that?


In what way are they nested?  The problems happen only with overlays
that have the 'display' property.


Well, only the outermost have a display property, so I guess it doesn’t
apply.

That said, your change seems to have fixed it.


Thanks for testing.  I must ask, though: why do you need to use
display strings (rather than, say, invisible property) to do folding?


Because I like to have it show how many lines are hidden.  So it’s not
necessary, but nice.

-Ivan

[-- Attachment #2: Type: text/html, Size: 3833 bytes --]