From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#35453: 26.1; Poor performance of vertical-motion in large org buffer Date: Sun, 28 Apr 2019 18:20:09 +0300 Message-ID: <83zhoatavq.fsf@gnu.org> References: <87pnpaob79.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="95287"; mail-complaints-to="usenet@blaine.gmane.org" Cc: 35453@debbugs.gnu.org To: 'Ihor Radchenko' Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Apr 28 17:21:11 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hKlc6-000Ogk-Vs for geb-bug-gnu-emacs@m.gmane.org; Sun, 28 Apr 2019 17:21:11 +0200 Original-Received: from localhost ([127.0.0.1]:45398 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hKlc5-0007YC-S4 for geb-bug-gnu-emacs@m.gmane.org; Sun, 28 Apr 2019 11:21:09 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:37615) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hKlbz-0007Y6-Iy for bug-gnu-emacs@gnu.org; Sun, 28 Apr 2019 11:21:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hKlby-0000FV-Fr for bug-gnu-emacs@gnu.org; Sun, 28 Apr 2019 11:21:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:52311) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hKlby-0000FN-Bv for bug-gnu-emacs@gnu.org; Sun, 28 Apr 2019 11:21:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hKlby-0003td-2u for bug-gnu-emacs@gnu.org; Sun, 28 Apr 2019 11:21:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 28 Apr 2019 15:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35453 X-GNU-PR-Package: emacs Original-Received: via spool by 35453-submit@debbugs.gnu.org id=B35453.155646482914920 (code B ref 35453); Sun, 28 Apr 2019 15:21:02 +0000 Original-Received: (at 35453) by debbugs.gnu.org; 28 Apr 2019 15:20:29 +0000 Original-Received: from localhost ([127.0.0.1]:37622 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hKlbQ-0003sa-WE for submit@debbugs.gnu.org; Sun, 28 Apr 2019 11:20:29 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:51288) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hKlbP-0003sP-7c for 35453@debbugs.gnu.org; Sun, 28 Apr 2019 11:20:27 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:37024) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hKlbK-0008Nn-28; Sun, 28 Apr 2019 11:20:22 -0400 Original-Received: from [176.228.60.248] (port=4484 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hKlbJ-0000Bz-1O; Sun, 28 Apr 2019 11:20:21 -0400 In-reply-to: <87pnpaob79.fsf@yantar92-laptop.i-did-not-set--mail-host-address--so-tickle-me> (message from 'Ihor Radchenko' on Thu, 25 Apr 2019 14:23:06 +0800) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:158394 Archived-At: > From: 'Ihor Radchenko' > Date: Thu, 25 Apr 2019 14:23:06 +0800 > > C-n and C-p are extremely slow (>10 sec to move one visual line) when > moving around a large org buffer in overview state, especially right > after opening the file. > The issue seems to be with low-level vertical-motion command - M-: > (vertical-motion 1) also takes >10sec on some lines. > > Steps to reproduce (emacs -Q): > 1. Open the attached org file > 2. C-n, C-n, C-n, C-n > 3. Observe emacs hanging for >10sec. I'm not sure I understand: is your problem with vertical-motion, or in general with moving by lines in that file's buffer? If the problem is the latter, then my advice to Org users is to use "C-c C-n/C-p" and "C-c C-f/C-b", not the normal cursor motion commands, because Org makes the latter very slow when a large portion of a large buffer is hidden. I will now try to explain why. This buffer's size is around 1MB and 42K lines, and the part between the 3rd and the 4th heading holds its lion's share: almost 40K lines of text. When C-n calls vertical-motion, the latter needs to find the buffer position displayed directly below the place where you typed C-n. Since much of the text between these places, vertical-motion needs to skip the invisible text as quickly as possible, because from the user's POV that text "doesn't exist": it isn't on the screen. However, Org makes this skipping exceedingly hard, because (1) it uses overlays (as opposed to text properties) to hide text, and (2) it puts an awful lot of overlays on the hidden text: there are 18400 overlays in this file's buffer, 17500 of them between the 3rd and the 4th heading. Because of this, vertical-motion must examine each and every overlay as it moves through the text, because each overlay can potentially change the 'invisible' property of text, or it might have a display string that needs to be displayed. So instead of skipping all that hidden text in one go, vertical-motion loops over those 17.5K overlays examining the properties of each one of them. And that takes time. Profiling shows that 80%(!) of the CPU time is spent in the function overlays_at which looks for overlays at a given buffer position, and 10% more in marker_position (because overlay endpoints are markers). So 90% of the time you wait for the cursor to move is spent processing overlays. Compare this with Outline mode, which places a single overlay on the entire hidden text between headings. If you visit the same file in Outline mode, C-n will be much, much faster. So with the current implementation of Org and overlays, Org users are well advised not to make such large bodies in their Org files, and if they do, definitely not to use C-n and C-p for vertical motion. Of course, if someone comes up with ideas how to speed up vertical-motion without changing what Org does with overlays and/or how overlays are implemented, such ideas will be most welcome.