From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ivan Andrus Newsgroups: gmane.emacs.bugs Subject: bug#17244: 24.3.90; `line-move-visual' errors when moving across wrapped lines with an overlay property of 'display Date: Mon, 14 Apr 2014 10:12:14 -0600 Message-ID: References: <834n1yu6lz.fsf@gnu.org> <0B5D9042-8250-4B40-9445-CEC864CFF458@gmail.com> <83fvlgl495.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=047d7b86e2105859e704f702f2b9 X-Trace: ger.gmane.org 1397492004 28781 80.91.229.3 (14 Apr 2014 16:13:24 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 14 Apr 2014 16:13:24 +0000 (UTC) Cc: 17244@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Apr 14 18:13:18 2014 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1WZjVU-000085-Cg for geb-bug-gnu-emacs@m.gmane.org; Mon, 14 Apr 2014 18:13:16 +0200 Original-Received: from localhost ([::1]:44070 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WZjVT-0005gp-Vj for geb-bug-gnu-emacs@m.gmane.org; Mon, 14 Apr 2014 12:13:15 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:41445) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WZjVL-0005ZJ-VE for bug-gnu-emacs@gnu.org; Mon, 14 Apr 2014 12:13:12 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WZjVH-0008Ly-DM for bug-gnu-emacs@gnu.org; Mon, 14 Apr 2014 12:13:07 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:39926) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WZjVH-0008Lo-9y for bug-gnu-emacs@gnu.org; Mon, 14 Apr 2014 12:13:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1WZjVG-0000Lq-I1 for bug-gnu-emacs@gnu.org; Mon, 14 Apr 2014 12:13:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Ivan Andrus Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 14 Apr 2014 16:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 17244 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 17244-submit@debbugs.gnu.org id=B17244.13974919441291 (code B ref 17244); Mon, 14 Apr 2014 16:13:02 +0000 Original-Received: (at 17244) by debbugs.gnu.org; 14 Apr 2014 16:12:24 +0000 Original-Received: from localhost ([127.0.0.1]:48083 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1WZjUd-0000Kk-Rq for submit@debbugs.gnu.org; Mon, 14 Apr 2014 12:12:24 -0400 Original-Received: from mail-pd0-f180.google.com ([209.85.192.180]:40105) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1WZjUa-0000KP-FD for 17244@debbugs.gnu.org; Mon, 14 Apr 2014 12:12:21 -0400 Original-Received: by mail-pd0-f180.google.com with SMTP id v10so8250714pde.39 for <17244@debbugs.gnu.org>; Mon, 14 Apr 2014 09:12:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yYTJBN1goxH2IHJ3kMxli84LW0TxqTh0x0P2tblJIXc=; b=nWy6PKzzdpKkg8pU/9KEkSya1rQX7eesaHt9XmtzZeFn3Kxcne4BkQdHaDkC5qOfeE guZe0Ng3F2ebQjgAjdPSIXKZv8hxx6K7w0tqFRhzI+7qBURq7Ddh0ysHwzdldhUINbuP x8Zqo69YhvV/PbH/yXSaGyYEJ3ybcKm56gpeFFwZECScRx1YSSRZDk3zezgQYli8ykyQ N3EahtRS0H9skH7N+yVk8uFv13PTqNKkMGOJrUl3eL99+XFDwWvcNeTjV9DMnwgbW493 YeAF0sjOdRo3OhA2z5fCiK27qSc7PGc9m68DdMATK6gmwEHzmeFp1h+qC8E0NdkgehtJ j+RA== X-Received: by 10.66.153.80 with SMTP id ve16mr3968480pab.143.1397491934712; Mon, 14 Apr 2014 09:12:14 -0700 (PDT) Original-Received: by 10.70.17.101 with HTTP; Mon, 14 Apr 2014 09:12:14 -0700 (PDT) In-Reply-To: <83fvlgl495.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:88015 Archived-At: --047d7b86e2105859e704f702f2b9 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Apr 14, 2014, at 2:11 AM, Eli Zaretskii wrote: From: Ivan Andrus Date: Sun, 13 Apr 2014 19:43:08 -0600 Cc: 17244-done@debbugs.gnu.org To tell the truth, Lisp code which covers large portions of buffer text with much shorter display strings that include newlines deserves to be broken. The current Emacs display engine was never designed to handle situations where the displayed text is so starkly different from buffer text, so the result of trying to fix "bugs" such as this one is a never-ending series of band-aids, one upon the other, which make the code utterly incomprehensible and unmaintainable. So I'm this close to refusing to fix such "bugs", and instead asking the authors of such Lisp to either find more benign ways of expressing what they need, or work around the limitations of the display engine in their own Lisp. So is the problem having newlines in the 'display property, or hiding large portions of the buffer? Each one of these is a problem. When they are present together in the same display string, the problems grow exponentially. Because, in my original use case (my modified version of http://www.emacswiki.org/emacs/fold.el) I don=E2=80=99t think I have newlin= es in the display property, though I definitely hide large portions of the buffer= . This is enough to cause problems. Consider what Emacs needs to do when you invoke "C-u N C-p" or "(line-move-visual -N)". It needs to find out what is the buffer position N screen lines before point. But where in the buffer is that? When those N lines display what is mostly buffer text, we can go back the appropriate amount of physical lines, and then start looking from there. But if what's on display in those N lines has little if any resemblance to what's in the buffer, the only safe algorithm is to go to the beginning of the buffer and start from there, which is prohibitively expensive. So Emacs implements several optimizations to make the operations reasonably fast. These optimizations only succeed when the display strings and the text they cover are approximately similar in length, or if the display strings are very short. The specific optimization that caused this bug was an attempt to be faster when text lines in a buffer are very long. I see. Thanks for explaining. The overlays are also nested, so it might it be related to that? In what way are they nested? The problems happen only with overlays that have the 'display' property. Well, only the outermost have a display property, so I guess it doesn=E2=80= =99t apply. That said, your change seems to have fixed it. Thanks for testing. I must ask, though: why do you need to use display strings (rather than, say, invisible property) to do folding? Because I like to have it show how many lines are hidden. So it=E2=80=99s = not necessary, but nice. -Ivan --047d7b86e2105859e704f702f2b9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Apr 14= , 2014, at 2:11 AM, Eli Zaretskii <eliz@gnu.org> wrote:

<= blockquote type=3D"cite"> From: Ivan Andrus <darthandrus@gmail.com>
Date: Sun, 13 Apr 2014 19:43:08 -060= 0
Cc: 17= 244-done@debbugs.gnu.org

<rant>
To tell the truth, Lisp code = which covers large portions of buffer
text with much shorter display str= ings that include newlines deserves
to be broken. =C2=A0The current Emac= s display engine was never designed to
handle situations where the displayed text is so starkly different
from = buffer text, so the result of trying to fix "bugs" such as thisone is a never-ending series of band-aids, one upon the other, which
make the code utterly incomprehensible and unmaintainable.

So I'= m this close to refusing to fix such "bugs", and instead askingthe authors of such Lisp to either find more benign ways of expressing what they need, or work around the limitations of the display engine
in = their own Lisp.
</rant>

So is the problem having newlines in the 'displa= y property, or hiding large portions of the buffer?

Each one of these is a problem. =C2=A0When they are presen= t together in the
same display string, the problems grow exponentially.<= br>
Because, in my original use case (my modif= ied version of http://www.emacswiki.org/emacs/fold.el) I don=E2=80=99t think = I have newlines in the display property, though I definitely hide large por= tions of the buffer.

This is enough to cause problems. =C2=A0Consider what Emac= s needs to do
when you invoke "C-u N C-p" or "(line-move-= visual -N)". =C2=A0It needs to
find out what is the buffer position= N screen lines before point. =C2=A0But
where in the buffer is that? =C2=A0When those N lines display what is
mo= stly buffer text, we can go back the appropriate amount of physical
line= s, and then start looking from there. =C2=A0But if what's on display in=
those N lines has little if any resemblance to what's in the buffer= ,
the only safe algorithm is to go to the beginning of the buffer and
star= t from there, which is prohibitively expensive. =C2=A0So Emacs
implement= s several optimizations to make the operations reasonably
fast. =C2=A0Th= ese optimizations only succeed when the display strings and
the text they cover are approximately similar in length, or if the
displ= ay strings are very short. =C2=A0The specific optimization that caused
t= his bug was an attempt to be faster when text lines in a buffer are
very= long.

I see. =C2=A0Thanks for explaining.

The overlays are also nested, so it mi= ght it be related to that?

In what way are they nested?= =C2=A0The problems happen only with overlays
that have the 'display' property.

Well, only th= e outermost have a display property, so I guess it doesn=E2=80=99t apply.
That said, your c= hange seems to have fixed it. =C2=A0

Thanks for testing. =C2=A0I must ask, though: why do you n= eed to use
display strings (rather than, say, invisible property) to do = folding?

Because I like to have it show how many lines = are hidden. =C2=A0So it=E2=80=99s not necessary, but nice.

-Ivan
--047d7b86e2105859e704f702f2b9--