unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* follow-mode: extremely slow in combination with org-mode
@ 2018-06-16 10:25 Gerald Wildgruber
  2018-06-16 11:41 ` Eli Zaretskii
  2018-06-21  8:25 ` Anders Lindgren
  0 siblings, 2 replies; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-16 10:25 UTC (permalink / raw)
  To: emacs-devel


Hi,

I have got a problem with enormous lag while entering text in buffers
with emacs follow-mode enabled.

I'm using emacs (git checkout v. 27.0.50) and Org mode (git checkout
release_9.1.13-760-g8def68).

My work is solely text-based, using org-mode. A typical setup is to use
a maximized or full screen emacs frame split into five windows
positioned vertically one next to the other on a 40" 4k display. All
five windows display one and the same file, that is opened using emacs
follow-mode, so that every window is displaying a portion of the same
file in a continuous manner (simultaneously displaying up to 45kb of
text in one frame, an entire paper).

Unfortunately, with this setup there is terrible lag with every single
key input (on a very fast quad core machine); every key stroke produces
a 100% processor load.

If I deactivate follow-mode, the problem disappears. It also gets better
if I enable text-mode instead of org-mode.

I then used the elisp profiler (M-x profiler-start/report) to find out
which function uses most cpu time while editing text in said setup. Here
is the result:

Collapsed, the report looks like that:

+ command-execute                                                8789  47%
+ follow-post-command-hook                                       7755  41%
+ ...                                                            1976  10%
+ redisplay_internal (C function)                                 104   0%
+ yas--post-command-handler                                        40   0%
+ timer-event-handler                                              20   0%
  tooltip-hide                                                      7   0%


And somewhat expanded:

- follow-post-command-hook                                       7755  41%
 - follow-adjust-window                                          7755  41%
  - follow-windows-start-end                                     7732  41%
   - follow-calc-win-end                                         7732  41%
    + pos-visible-in-window-p                                      25   0%
    + posn-at-x-y                                                   7   0%
    + window-inside-pixel-edges                                     3   0%
  + follow-all-followers                                            4   0%
    follow-avoid-tail-recenter                                      3   0%

If I understand correctly "follow-calc-win-end" would be the function
that uses most of cpu time.

I then did additional profiling with the elp library, adding relevant
functions under "Command-execute" and "follow-post-command-hook" to
elp-function-list, edited text, and then did M-x elp-results. It showed
again that "follow-calc-win-end" by far used most of cpu time.

Anyone got an idea what's going on here and how to debug that? Are there
possible optimizations with this situation? Or is this "normal",
expected behavior, simply due to the number of windows and text
displayed?

Thanks

Gerald.

---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-16 10:25 follow-mode: extremely slow in combination with org-mode Gerald Wildgruber
@ 2018-06-16 11:41 ` Eli Zaretskii
  2018-06-16 13:06   ` Gerald Wildgruber
  2018-06-21  8:25 ` Anders Lindgren
  1 sibling, 1 reply; 18+ messages in thread
From: Eli Zaretskii @ 2018-06-16 11:41 UTC (permalink / raw)
  To: Gerald Wildgruber; +Cc: emacs-devel

> From: Gerald Wildgruber <wildgruber@tu-berlin.de>
> Date: Sat, 16 Jun 2018 12:25:13 +0200
> 
> - follow-post-command-hook                                       7755  41%
>  - follow-adjust-window                                          7755  41%
>   - follow-windows-start-end                                     7732  41%
>    - follow-calc-win-end                                         7732  41%
>     + pos-visible-in-window-p                                      25   0%
>     + posn-at-x-y                                                   7   0%
>     + window-inside-pixel-edges                                     3   0%
>   + follow-all-followers                                            4   0%
>     follow-avoid-tail-recenter                                      3   0%
> 
> If I understand correctly "follow-calc-win-end" would be the function
> that uses most of cpu time.

Yes, it looks like that.  For even more detailed results, repeat the
experiment after loading the uncompiled follow.el.

> Anyone got an idea what's going on here and how to debug that? Are there
> possible optimizations with this situation? Or is this "normal",
> expected behavior, simply due to the number of windows and text
> displayed?

If you "show all" in the Org buffer, do the lags become significantly
smaller?



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-16 11:41 ` Eli Zaretskii
@ 2018-06-16 13:06   ` Gerald Wildgruber
  2018-06-16 16:25     ` Eli Zaretskii
  0 siblings, 1 reply; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-16 13:06 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel


On Sa, Jun 16 2018, Eli Zaretskii <eliz@gnu.org> wrote:

>> If I understand correctly "follow-calc-win-end" would be the function
>> that uses most of cpu time.
>
> Yes, it looks like that.  For even more detailed results, repeat the
> experiment after loading the uncompiled follow.el.

Thanks Eli,

So what I did was:

1. M-x load-file RET path-to-follow.el.gz RET
2. M-x profiler start
3. edit text
4. M-x profiler-report

Results do indeed look slightly different:

- follow-post-command-hook                                     140786  88%
 - if                                                          140440  88%
  - let                                                        140440  88%
   - save-current-buffer                                       140440  88%
    - follow-adjust-window                                     140436  88%
     - if                                                      140436  88%
      - progn                                                  140436  88%
       - let                                                   140418  88%
        - let*                                                 140340  88%
         - if                                                  140327  87%
          - follow-redisplay                                   114860  72%
           - let*                                              114860  72%
            - cond                                             104536  65%
             - follow-calculate-first-window-start-from-above             104536  65%
              - save-excursion                                 104536  65%
               - let                                           104536  65%
                - while                                        104536  65%
                 - if                                          104536  65%
                  - setq                                       103381  64%
                   - follow-calc-win-start                     103381  64%
                    - while                                    103381  64%
                     - let                                     103378  64%
                      - setq                                   103349  64%
                       - car                                   103349  64%
                        - follow-calc-win-end                  103346  64%
                         - apply                               103346  64%
                          - #<compiled 0x226e139>              103329  64%
                           - apply                             103322  64%
                            - #<lambda 0xe5f44dcc127a05e>             103313  64%
                             - let*                            103310  64%
                                posn-point                      51699  32%
                              + if                              51546  32%
                              + window-inside-pixel-edges                 47   0%
                                +                                   3   0%
                        set-window-start                            4   0%
                     + and                                          3   0%
                  + not                                          1155   0%
            + let                                                4361   2%
            + follow-calc-win-start                              3289   2%
            + follow-estimate-first-window-start                 2674   1%
          - setq                                                20770  13%
           - follow-windows-start-end                           20770  13%
            - if                                                20770  13%
             - let                                              20770  13%
              - let                                             20770  13%
               - while                                          20770  13%
                - setq                                          20770  13%
                 - cons                                         20770  13%
                  - cons                                        20770  13%
                   - cons                                       20770  13%
                    - follow-calc-win-end                       20767  13%
                     - apply                                    20767  13%
                      + #<compiled 0x226e139>                   20763  13%
          + let                                                  4635   2%
          + if                                                     62   0%
         + progn                                                   13   0%
        + if                                                       67   0%
        + follow-all-followers                                     11   0%
       + follow-avoid-tail-recenter                                18   0%
    + if                                                            4   0%
 + follow-adjust-window                                           346   0%
+ command-execute                                                9690   6%

> If you "show all" in the Org buffer, do the lags become significantly
> smaller?

Yes, the difference is VERY significant; but I guess this is because
with collapsed headings, there is A LOT more text within the boundaries
of my frame, than with show-all.


Gerald.

---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-16 13:06   ` Gerald Wildgruber
@ 2018-06-16 16:25     ` Eli Zaretskii
  2018-06-16 21:05       ` Gerald Wildgruber
  2018-06-17  6:57       ` Gerald Wildgruber
  0 siblings, 2 replies; 18+ messages in thread
From: Eli Zaretskii @ 2018-06-16 16:25 UTC (permalink / raw)
  To: Gerald Wildgruber; +Cc: emacs-devel

> From: Gerald Wildgruber <wildgruber@tu-berlin.de>
> CC: <emacs-devel@gnu.org>
> Date: Sat, 16 Jun 2018 15:06:49 +0200
> 
> 
>                    - follow-calc-win-start                     103381  64%
>                     - while                                    103381  64%
>                      - let                                     103378  64%
>                       - setq                                   103349  64%
>                        - car                                   103349  64%
>                         - follow-calc-win-end                  103346  64%
>                          - apply                               103346  64%
>                           - #<compiled 0x226e139>              103329  64%
>                            - apply                             103322  64%
>                             - #<lambda 0xe5f44dcc127a05e>             103313  64%
>                              - let*                            103310  64%
>                                 posn-point                      51699  32%
>                               + if                              51546  32%
>                               + window-inside-pixel-edges                 47   0%

There's something here I don't understand.  This profile says that
follow-calc-win-end calls posn-point via two 'apply' calls, a
byte-compiled function, and a lambda-function.  But there's nothing
like that in follow.el that I could see.  Did you advise some
functions in follow.el?  If not, what are those intermediate calls?

> > If you "show all" in the Org buffer, do the lags become significantly
> > smaller?
> 
> Yes, the difference is VERY significant; but I guess this is because
> with collapsed headings, there is A LOT more text within the boundaries
> of my frame, than with show-all.

How much is "A LOT more"?  How many physical lines do you have in
typical single window in that Org buffer?



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-16 16:25     ` Eli Zaretskii
@ 2018-06-16 21:05       ` Gerald Wildgruber
  2018-06-17  6:57       ` Gerald Wildgruber
  1 sibling, 0 replies; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-16 21:05 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel


On Sa, Jun 16 2018, Eli Zaretskii <eliz@gnu.org> wrote:

> There's something here I don't understand.  This profile says that
> follow-calc-win-end calls posn-point via two 'apply' calls, a
> byte-compiled function, and a lambda-function.  But there's nothing
> like that in follow.el that I could see.  Did you advise some
> functions in follow.el?  If not, what are those intermediate calls?

No definitely not: I didn't modify any of the functions. But I will
repeat the profiling to see, if I get the same results or If I made an
error.

Gerald.

---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-16 16:25     ` Eli Zaretskii
  2018-06-16 21:05       ` Gerald Wildgruber
@ 2018-06-17  6:57       ` Gerald Wildgruber
  2018-06-17  8:12         ` Eli Zaretskii
  1 sibling, 1 reply; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-17  6:57 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel


On Sa, Jun 16 2018, Eli Zaretskii <eliz@gnu.org> wrote:

> There's something here I don't understand.  This profile says that
> follow-calc-win-end calls posn-point via two 'apply' calls, a
> byte-compiled function, and a lambda-function.  But there's nothing
> like that in follow.el that I could see.  Did you advise some
> functions in follow.el?  If not, what are those intermediate calls?

I repeated the profiling two times; the intermediate calls disappeared;
I can't say what might have caused them; here is the result:

- follow-post-command-hook                                      21938  71%
 - if                                                           21938  71%
  - let                                                         21938  71%
   - save-current-buffer                                        21938  71%
    - follow-adjust-window                                      21933  71%
     - if                                                       21933  71%
      - progn                                                   21933  71%
       - let                                                    21906  71%
        - let*                                                  21906  71%
         - progn                                                17609  57%
          - follow-windows-start-end                            17606  57%
           - if                                                 17606  57%
            - let                                               17603  57%
             - let                                              17603  57%
              - while                                           17603  57%
               - setq                                           17599  57%
                - cons                                          17599  57%
                 - cons                                         17599  57%
                  - cons                                        17599  57%
                   - follow-calc-win-end                        17591  57%
                    - let*                                      17591  57%
                     - if                                        9229  30%
                      + let                                      1115   3%
                      + pos-visible-in-window-p                    26   0%
                       posn-point                                8350  27%
                     + window-inside-pixel-edges                   12   0%
            + follow-cache-valid-p                                  3   0%
          + follow-update-window-start                              3   0%
         - if                                                    4297  13%
          - follow-redisplay                                     3329  10%
           - let*                                                3329  10%
            - cond                                               2924   9%
             - follow-calculate-first-window-start-from-above               2924   9%
              - save-excursion                                   2924   9%
               - let                                             2924   9%
                - while                                          2924   9%
                 - if                                            2924   9%
                  - setq                                         2899   9%
                   - follow-calc-win-start                       2899   9%
                    - while                                      2899   9%
                     - let                                       2899   9%
                      - setq                                     2896   9%
                       - car                                     2896   9%
                        - follow-calc-win-end                    2896   9%
                         - let*                                  2896   9%
                            posn-point                           1455   4%
                            if                                   1441   4%
                        set-window-start                            3   0%
                  + not                                            25   0%
            + let                                                 169   0%
            + follow-calc-win-start                               125   0%
            + follow-estimate-first-window-start                  111   0%
          + let                                                   598   1%
          + setq                                                  364   1%
          + follow-avoid-tail-recenter                              3   0%
          + and                                                     3   0%
       + follow-avoid-tail-recenter                                27   0%
      set-buffer                                                    5   0%


>> > If you "show all" in the Org buffer, do the lags become significantly
>> > smaller?
>>
>> Yes, the difference is VERY significant; but I guess this is because
>> with collapsed headings, there is A LOT more text within the boundaries
>> of my frame, than with show-all.
>
> How much is "A LOT more"?  How many physical lines do you have in
> typical single window in that Org buffer?

When I do a SHOW-ALL, my emacs frame split into those five windows
connected by follow-mode show about 700 lines of (natural language)
text. If COLLAPSED, it can be up to several thousand lines of text
hidden under the org headings (typically two to ten thousand lines).

Gerald.

---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17  6:57       ` Gerald Wildgruber
@ 2018-06-17  8:12         ` Eli Zaretskii
  2018-06-17 11:13           ` Alan Mackenzie
  0 siblings, 1 reply; 18+ messages in thread
From: Eli Zaretskii @ 2018-06-17  8:12 UTC (permalink / raw)
  To: Gerald Wildgruber, Alan Mackenzie; +Cc: emacs-devel

> From: Gerald Wildgruber <wildgruber@tu-berlin.de>
> CC: <emacs-devel@gnu.org>
> Date: Sun, 17 Jun 2018 08:57:22 +0200
> 
>           - follow-windows-start-end                            17606  57%
>            - if                                                 17606  57%
>             - let                                               17603  57%
>              - let                                              17603  57%
>               - while                                           17603  57%
>                - setq                                           17599  57%
>                 - cons                                          17599  57%
>                  - cons                                         17599  57%
>                   - cons                                        17599  57%
>                    - follow-calc-win-end                        17591  57%
>                     - let*                                      17591  57%
>                      - if                                        9229  30%
>                       + let                                      1115   3%
>                       + pos-visible-in-window-p                    26   0%
>                        posn-point                                8350  27%
>                      + window-inside-pixel-edges                   12   0%

Alan, why doesn't follow-calc-win-end doesn't just use window-end with
last argument non-nil?  AFAU, that should allow you to compute the
result using 1/4 of the processing time you now invest.  Why do you
need all the rest in that function?  What am I missing?



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17  8:12         ` Eli Zaretskii
@ 2018-06-17 11:13           ` Alan Mackenzie
  2018-06-17 13:02             ` Eli Zaretskii
  2018-06-17 15:13             ` Eli Zaretskii
  0 siblings, 2 replies; 18+ messages in thread
From: Alan Mackenzie @ 2018-06-17 11:13 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Gerald Wildgruber, emacs-devel

Hello, Eli.

On Sun, Jun 17, 2018 at 11:12:39 +0300, Eli Zaretskii wrote:
> > From: Gerald Wildgruber <wildgruber@tu-berlin.de>
> > CC: <emacs-devel@gnu.org>
> > Date: Sun, 17 Jun 2018 08:57:22 +0200

> >           - follow-windows-start-end                            17606  57%
> >            - if                                                 17606  57%
> >             - let                                               17603  57%
> >              - let                                              17603  57%
> >               - while                                           17603  57%
> >                - setq                                           17599  57%
> >                 - cons                                          17599  57%
> >                  - cons                                         17599  57%
> >                   - cons                                        17599  57%
> >                    - follow-calc-win-end                        17591  57%
> >                     - let*                                      17591  57%
> >                      - if                                        9229  30%
> >                       + let                                      1115   3%
> >                       + pos-visible-in-window-p                    26   0%
> >                        posn-point                                8350  27%
> >                      + window-inside-pixel-edges                   12   0%

> Alan, why doesn't follow-calc-win-end doesn't just use window-end with
> last argument non-nil?  AFAU, that should allow you to compute the
> result using 1/4 of the processing time you now invest.  Why do you
> need all the rest in that function?  What am I missing?

I honestly don't know for sure.  But on 2014-04-27 I made a note in my
personal log:

    "In follow-calc-win-end, we check for EOB being visible in the
    window, rather than being the window-end.  There needs to be a
    comment here about what the last element means."

.  I wish I could decipher what I meant, then.  I think I might have
encountered a corner case, where EOB is _inside_ a window, possibly not
even the last window, and using window-end failed for some reason.

But I think follow-calc-win-end was in the original follow.el written by
Anders Lindgren.  Maybe, back in the mists of time, window-end didn't
work for some reason - maybe it hadn't yet acquired it's UPDATE
argument, or something like that.

follow-calc-win-end was much longer in Emacs-21.4, and carried the
comment "This function is optimized function for speed!".  It had got
shorter some time up to Emacs-24.5, where it had already started using
posn-point.  This rewrite was probably this:

    commit c93b886f957b55df4fe45d986c34242832ec4a28
    Author: Chong Yidong <cyd@gnu.org>
    Date:   Sun Apr 29 09:48:23 2012 +0800

        Fix Follow mode's calculation of window ends.

        * lisp/follow.el (follow-calc-win-end): Rewrite to handle partial
        screen lines correctly.
        (follow-avoid-tail-recenter): Minor cleanup.

        Fixes: debbugs:8390

Why we don't just use window-end, I don't know.  Maybe the sequence of
people who've modified follow-calc-win-end have all assumed their
predecessors understood why a simple window-end wasn't used.  If it is
something to do with EOB messing things up, maybe we could amend
window-end better to handle EOB.

Sorry I can't do any more than speculate, here.

-- 
Alan Mackenzie (Nuremberg, Germany).



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17 11:13           ` Alan Mackenzie
@ 2018-06-17 13:02             ` Eli Zaretskii
  2018-06-17 14:56               ` Gerald Wildgruber
  2018-06-17 15:13             ` Eli Zaretskii
  1 sibling, 1 reply; 18+ messages in thread
From: Eli Zaretskii @ 2018-06-17 13:02 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: wildgruber, emacs-devel

> Date: Sun, 17 Jun 2018 11:13:33 +0000
> Cc: Gerald Wildgruber <wildgruber@tu-berlin.de>, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> On Sun, Jun 17, 2018 at 11:12:39 +0300, Eli Zaretskii wrote:
> > > From: Gerald Wildgruber <wildgruber@tu-berlin.de>
> > > CC: <emacs-devel@gnu.org>
> > > Date: Sun, 17 Jun 2018 08:57:22 +0200
> 
> > >           - follow-windows-start-end                            17606  57%
> > >            - if                                                 17606  57%
> > >             - let                                               17603  57%
> > >              - let                                              17603  57%
> > >               - while                                           17603  57%
> > >                - setq                                           17599  57%
> > >                 - cons                                          17599  57%
> > >                  - cons                                         17599  57%
> > >                   - cons                                        17599  57%
> > >                    - follow-calc-win-end                        17591  57%
> > >                     - let*                                      17591  57%
> > >                      - if                                        9229  30%
> > >                       + let                                      1115   3%
> > >                       + pos-visible-in-window-p                    26   0%
> > >                        posn-point                                8350  27%
> > >                      + window-inside-pixel-edges                   12   0%
> 
> > Alan, why doesn't follow-calc-win-end doesn't just use window-end with
> > last argument non-nil?  AFAU, that should allow you to compute the
> > result using 1/4 of the processing time you now invest.  Why do you
> > need all the rest in that function?  What am I missing?
> 
> I honestly don't know for sure.  But on 2014-04-27 I made a note in my
> personal log:
> 
>     "In follow-calc-win-end, we check for EOB being visible in the
>     window, rather than being the window-end.  There needs to be a
>     comment here about what the last element means."
> 
> .  I wish I could decipher what I meant, then.  I think I might have
> encountered a corner case, where EOB is _inside_ a window, possibly not
> even the last window, and using window-end failed for some reason.
> 
> But I think follow-calc-win-end was in the original follow.el written by
> Anders Lindgren.  Maybe, back in the mists of time, window-end didn't
> work for some reason - maybe it hadn't yet acquired it's UPDATE
> argument, or something like that.

OK, thanks for the explanations.

Gerald, could you try the following patch to follow-calc-win-end?
Please first see if this makes it significantly faster in your use
cases, and if so, please run with it for a while and see if there are
some adverse effects.  (If this is not faster, it isn't worth testing
it more.)

diff --git a/lisp/follow.el b/lisp/follow.el
index fd397c0..c99fc93 100644
--- a/lisp/follow.el
+++ b/lisp/follow.el
@@ -925,14 +925,17 @@ follow-calc-win-end
 the last fully-visible line in WIN.  END-OF-BUFFER is t when EOB
 is fully-visible in WIN.  If WIN is nil, the selected window is
 used."
-  (let* ((win (or win (selected-window)))
-	 (edges (window-inside-pixel-edges win))
-	 (ht (- (nth 3 edges) (nth 1 edges)))
-	 (last-line-pos (posn-point (posn-at-x-y 0 (1- ht) win))))
-    (if (pos-visible-in-window-p last-line-pos win)
-	(let ((end (window-end win t)))
-	  (list end (pos-visible-in-window-p (point-max) win)))
-      (list last-line-pos nil))))
+  (or win (setq win (selected-window)))
+  (with-selected-window win
+    (let* ((wend (window-end win t))
+           (endp (= wend (point-max))))
+      (and (> wend (point-min))
+           (setq wend (1- wend)))
+      (setq wend
+            (save-excursion
+              (goto-char wend)
+              (line-beginning-position)))
+      (list wend endp))))
 
 (defun follow-calc-win-start (windows pos win)
   "Determine the start of window WIN in a Follow mode window chain.



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17 13:02             ` Eli Zaretskii
@ 2018-06-17 14:56               ` Gerald Wildgruber
  2018-06-17 15:10                 ` Eli Zaretskii
  0 siblings, 1 reply; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-17 14:56 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Alan Mackenzie, emacs-devel


Thanks again  Eli and Alan for picking this up!

Eli: the patch didn't apply cleanly (with "patch -p1 <" or "git apply"),
but as it is so simple I applied it manually.

follow-calc-win-end now looks like that:

==================================================

(defun follow-calc-win-end (&optional win)
  "Calculate the end position for window WIN.
Return (END-POS END-OF-BUFFER).

Actually, the position returned is the start of the line after
the last fully-visible line in WIN.  END-OF-BUFFER is t when EOB
is fully-visible in WIN.  If WIN is nil, the selected window is
used."
  (or win (setq win (selected-window)))
  (with-selected-window win
    (let* ((wend (window-end win t))
           (endp (= wend (point-max))))
      (and (> wend (point-min))
           (setq wend (1- wend)))
      (setq wend
            (save-excursion
              (goto-char wend)
              (line-beginning-position)))
      (list wend endp))))

==================================================

I byte compiled the patched follow.el restarted emacs and began editing.

It actually seems somewhat faster, though not enormously; I will
continue working with this patched version to see how it goes; I have a
more problematic (=slower) machine in my office, and I'm am eager to
see, if it makes a difference there; at home, where I'm testing right
now, its a new quad core xeon with 64gb of ram, -- and it is kind of
strange that with such a machine fans start very audibly as soon as you
start TYPING (not video editing or the like) :-)

One more observation: it might be that the slowing down becomes more
noticeable, the closer point gets to the bottom of the last window
(lower right edge of my five-fold splitted emacs frame), but I couldn't
reproduce this consistently.

Please count on me for any other testing that might be necessary! I'm no
programmer but will do my best to deliver usable results.

Here's how the profile looks now, with the patched, uncompiled follow.el
loaded right before profiling:

- follow-post-command-hook                                      17642  40%
 - if                                                           17642  40%
  - let                                                         17642  40%
   - save-current-buffer                                        17642  40%
    - follow-adjust-window                                      17642  40%
     - if                                                       17642  40%
      - progn                                                   17642  40%
       - let                                                    17591  40%
        - let*                                                  17560  40%
         - progn                                                16733  38%
          - follow-windows-start-end                            16730  38%
           - if                                                 16730  38%
            - let                                               16730  38%
             - let                                              16730  38%
              - while                                           16730  38%
               - setq                                           16719  38%
                - cons                                          16564  37%
                 - cons                                         16560  37%
                  - cons                                        16560  37%
                   - follow-calc-win-end                        16557  37%
                    - let                                       16557  37%
                     - save-current-buffer                      16557  37%
                      - unwind-protect                          16557  37%
                       - progn                                  16551  37%
                          let*                                  16548  37%
                         internal--after-with-selected-window                  3   0%
                + follow-windows-start-end                        155   0%
         + if                                                     827   1%
        + follow-all-followers                                     27   0%
       + follow-avoid-tail-recenter                                51   0%
+ command-execute                                               17015  38%


Gerald.




On So, Jun 17 2018, Eli Zaretskii <eliz@gnu.org> wrote:

> Gerald, could you try the following patch to follow-calc-win-end?
> Please first see if this makes it significantly faster in your use
> cases, and if so, please run with it for a while and see if there are
> some adverse effects.  (If this is not faster, it isn't worth testing
> it more.)
>
> diff --git a/lisp/follow.el b/lisp/follow.el
> index fd397c0..c99fc93 100644
> --- a/lisp/follow.el
> +++ b/lisp/follow.el
> @@ -925,14 +925,17 @@ follow-calc-win-end
>  the last fully-visible line in WIN.  END-OF-BUFFER is t when EOB
>  is fully-visible in WIN.  If WIN is nil, the selected window is
>  used."
> -  (let* ((win (or win (selected-window)))
> -	 (edges (window-inside-pixel-edges win))
> -	 (ht (- (nth 3 edges) (nth 1 edges)))
> -	 (last-line-pos (posn-point (posn-at-x-y 0 (1- ht) win))))
> -    (if (pos-visible-in-window-p last-line-pos win)
> -	(let ((end (window-end win t)))
> -	  (list end (pos-visible-in-window-p (point-max) win)))
> -      (list last-line-pos nil))))
> +  (or win (setq win (selected-window)))
> +  (with-selected-window win
> +    (let* ((wend (window-end win t))
> +           (endp (= wend (point-max))))
> +      (and (> wend (point-min))
> +           (setq wend (1- wend)))
> +      (setq wend
> +            (save-excursion
> +              (goto-char wend)
> +              (line-beginning-position)))
> +      (list wend endp))))
>
>  (defun follow-calc-win-start (windows pos win)
>    "Determine the start of window WIN in a Follow mode window chain.

---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17 14:56               ` Gerald Wildgruber
@ 2018-06-17 15:10                 ` Eli Zaretskii
  2018-06-19 12:08                   ` Gerald Wildgruber
  2018-06-19 13:53                   ` Gerald Wildgruber
  0 siblings, 2 replies; 18+ messages in thread
From: Eli Zaretskii @ 2018-06-17 15:10 UTC (permalink / raw)
  To: Gerald Wildgruber; +Cc: acm, emacs-devel

> From: Gerald Wildgruber <wildgruber@tu-berlin.de>
> CC: Alan Mackenzie <acm@muc.de>, <emacs-devel@gnu.org>
> Date: Sun, 17 Jun 2018 16:56:00 +0200
> 
> I byte compiled the patched follow.el restarted emacs and began editing.
> 
> It actually seems somewhat faster, though not enormously;

If the speedup is not significant, then I guess there are other
factors at work, although the profile seemed to show I was eliminating
the hottest function calls.

> I will continue working with this patched version to see how it
> goes; I have a more problematic (=slower) machine in my office, and
> I'm am eager to see, if it makes a difference there; at home, where
> I'm testing right now, its a new quad core xeon with 64gb of ram, --
> and it is kind of strange that with such a machine fans start very
> audibly as soon as you start TYPING (not video editing or the like)
> :-)

Each character you insert triggers redisplay, and I'm guessing
follow-mode has many of the redisplay optimizations disabled.  But it
may be interesting to see a profile when all you do is type
self-inserting characters, maybe we will learn something important
from that.

> One more observation: it might be that the slowing down becomes more
> noticeable, the closer point gets to the bottom of the last window
> (lower right edge of my five-fold splitted emacs frame), but I couldn't
> reproduce this consistently.

This could be due to follow-calc-win-start, which loops over windows.
Or maybe there's another loop.  Again, a profile might tell something
interesting.

> Please count on me for any other testing that might be necessary! I'm no
> programmer but will do my best to deliver usable results.
> 
> Here's how the profile looks now, with the patched, uncompiled follow.el
> loaded right before profiling:

Thanks.



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17 11:13           ` Alan Mackenzie
  2018-06-17 13:02             ` Eli Zaretskii
@ 2018-06-17 15:13             ` Eli Zaretskii
  1 sibling, 0 replies; 18+ messages in thread
From: Eli Zaretskii @ 2018-06-17 15:13 UTC (permalink / raw)
  To: Alan Mackenzie; +Cc: wildgruber, emacs-devel

> Date: Sun, 17 Jun 2018 11:13:33 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Gerald Wildgruber <wildgruber@tu-berlin.de>, emacs-devel@gnu.org
> 
>     commit c93b886f957b55df4fe45d986c34242832ec4a28
>     Author: Chong Yidong <cyd@gnu.org>
>     Date:   Sun Apr 29 09:48:23 2012 +0800
> 
>         Fix Follow mode's calculation of window ends.
> 
>         * lisp/follow.el (follow-calc-win-end): Rewrite to handle partial
>         screen lines correctly.
>         (follow-avoid-tail-recenter): Minor cleanup.
> 
>         Fixes: debbugs:8390

Btw, bug#8390 is back.  My testing indicates that it was fixed in
Emacs 24.3 (I guess by the above change), and then became broken again
in Emacs 24.4.  Try "C-x C--" in a buffer under follow-mode, and you
will see the erratic behavior.



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17 15:10                 ` Eli Zaretskii
@ 2018-06-19 12:08                   ` Gerald Wildgruber
  2018-06-19 16:55                     ` Eli Zaretskii
  2018-06-19 13:53                   ` Gerald Wildgruber
  1 sibling, 1 reply; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-19 12:08 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: acm, emacs-devel


Hi

I tried the patched follow.el (the one that contains the modified
follow-calc-win-end) on my somewhat slower (but also new quad core)
office machine, unfortunately no amelioration of lag was noticeable.

Same procedure: patching follow.el, starting the profiler, loading the
patched uncompiled version into emacs via M-x load-file and finally
start editing.

The lag remains enormous (e.g. more than a second lag between typing RET
and seeing a newline on screen); I noticed however, that upon typing RET
one time, the newline comes almost immediately; but after typing more
RETs they take ages to appear on screen.

Bizarrely, this happened also: redisplay takes very long and it
apparently aborts leaving paragraphs before point unfilled, one long
line of text (I always use auto-fill-mode).

On So, Jun 17 2018, Eli Zaretskii <eliz@gnu.org> wrote:

> Each character you insert triggers redisplay, and I'm guessing
> follow-mode has many of the redisplay optimizations disabled.

Actually, I had no idea. I tried to understand what "redisplay" is (by
reading the comments at the beginning of emacs/src/xdisp.c) and the more
I read, the less I am surprised it takes so long! :-)

Why might those optimizations have been disabled on follow-mode?

> But it may be interesting to see a profile when all you do is type
> self-inserting characters, maybe we will learn something important
> from that.

Here is another profile, now on my office machine with only
self-inserting commands, i.e. typing just "normal" letters:

- follow-post-command-hook                                      54982  61%
 - if                                                           54982  61%
  - let                                                         54975  61%
   - save-current-buffer                                        54975  61%
    - follow-adjust-window                                      54975  61%
     - if                                                       54972  61%
      - progn                                                   54972  61%
       - let                                                    53646  59%
        - let*                                                  53646  59%
         - if                                                   53646  59%
          - follow-redisplay                                    49062  54%
           - let*                                               49062  54%
            - cond                                              43568  48%
             - follow-calculate-first-window-start-from-above              43568  48%
              - save-excursion                                  43568  48%
               - let                                            43568  48%
                - while                                         43564  48%
                 - if                                           43564  48%
                  - setq                                        42813  47%
                   - follow-calc-win-start                      42813  47%
                    - while                                     42813  47%
                     - let                                      42802  47%
                      - setq                                    42789  47%
                       - car                                    42789  47%
                        - follow-calc-win-end                   42785  47%
                         - let                                  42785  47%
                          - save-current-buffer                 42785  47%
                           - unwind-protect                     42785  47%
                            - progn                             42777  47%
                             - let*                             42767  47%
                              + setq                               10   0%
                              + window-end                          4   0%
                     + and                                          7   0%
                  + not                                           747   0%
                    if                                              4   0%
            + follow-calc-win-start                              3062   3%
            + follow-estimate-first-window-start                 1410   1%
            + let                                                1022   1%
          - let                                                  4574   5%
           - redisplay                                           4570   5%
            - redisplay_internal (C function)                      17   0%
             - eval                                                11   0%
                if                                                  8   0%
                mode-line-eol-desc                                  3   0%
          + follow-avoid-tail-recenter                              7   0%
            if                                                      3   0%
       + follow-windows-start-end                                1326   1%
     + progn                                                        3   0%
  + follow-post-command-hook                                        3   0%
+ command-execute                                               27012  30%


Anything else to try to get a faster follow-mode?

Thanks

Gerald.


---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-17 15:10                 ` Eli Zaretskii
  2018-06-19 12:08                   ` Gerald Wildgruber
@ 2018-06-19 13:53                   ` Gerald Wildgruber
  1 sibling, 0 replies; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-19 13:53 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: acm, emacs-devel

Hi

I tried the patched follow.el (the one that contains the modified
follow-calc-win-end) on my somewhat slower (but also new quad core)
office machine, unfortunately no amelioration of lag was noticeable.

Same procedure: patching follow.el, starting the profiler, loading the
patched uncompiled version into emacs via M-x load-file and finally
start editing.

The lag remains enormous (e.g. more than a second lag between typing RET
and seeing a newline on screen); I noticed however, that upon typing RET
one time, the newline comes almost immediately; but after typing more
RETs they take ages to appear on screen.

Bizarrely, this happened also: redisplay takes very long and it
apparently aborts leaving paragraphs before point unfilled, one long
line of text (I always use auto-fill-mode).

On So, Jun 17 2018, Eli Zaretskii <eliz@gnu.org> wrote:

> Each character you insert triggers redisplay, and I'm guessing
> follow-mode has many of the redisplay optimizations disabled.

Actually, I had no idea. I tried to understand what "redisplay" is (by
reading the comments at the beginning of emacs/src/xdisp.c) and the more
I read, the less I am surprised it takes so long! :-)

Why might those optimizations have been disabled on follow-mode?

> But it may be interesting to see a profile when all you do is type
> self-inserting characters, maybe we will learn something important
> from that.

Here is another profile, now on my office machine with only
self-inserting commands, i.e. typing just "normal" letters:

- follow-post-command-hook                                      54982  61%
 - if                                                           54982  61%
  - let                                                         54975  61%
   - save-current-buffer                                        54975  61%
    - follow-adjust-window                                      54975  61%
     - if                                                       54972  61%
      - progn                                                   54972  61%
       - let                                                    53646  59%
        - let*                                                  53646  59%
         - if                                                   53646  59%
          - follow-redisplay                                    49062  54%
           - let*                                               49062  54%
            - cond                                              43568  48%
             - follow-calculate-first-window-start-from-above              43568  48%
              - save-excursion                                  43568  48%
               - let                                            43568  48%
                - while                                         43564  48%
                 - if                                           43564  48%
                  - setq                                        42813  47%
                   - follow-calc-win-start                      42813  47%
                    - while                                     42813  47%
                     - let                                      42802  47%
                      - setq                                    42789  47%
                       - car                                    42789  47%
                        - follow-calc-win-end                   42785  47%
                         - let                                  42785  47%
                          - save-current-buffer                 42785  47%
                           - unwind-protect                     42785  47%
                            - progn                             42777  47%
                             - let*                             42767  47%
                              + setq                               10   0%
                              + window-end                          4   0%
                     + and                                          7   0%
                  + not                                           747   0%
                    if                                              4   0%
            + follow-calc-win-start                              3062   3%
            + follow-estimate-first-window-start                 1410   1%
            + let                                                1022   1%
          - let                                                  4574   5%
           - redisplay                                           4570   5%
            - redisplay_internal (C function)                      17   0%
             - eval                                                11   0%
                if                                                  8   0%
                mode-line-eol-desc                                  3   0%
          + follow-avoid-tail-recenter                              7   0%
            if                                                      3   0%
       + follow-windows-start-end                                1326   1%
     + progn                                                        3   0%
  + follow-post-command-hook                                        3   0%
+ command-execute                                               27012  30%


Anything else to try to get a faster follow-mode?

Thanks

Gerald.


---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-19 12:08                   ` Gerald Wildgruber
@ 2018-06-19 16:55                     ` Eli Zaretskii
  0 siblings, 0 replies; 18+ messages in thread
From: Eli Zaretskii @ 2018-06-19 16:55 UTC (permalink / raw)
  To: Gerald Wildgruber; +Cc: acm, emacs-devel

> From: Gerald Wildgruber <wildgruber@tu-berlin.de>
> CC: <acm@muc.de>, <emacs-devel@gnu.org>
> Date: Tue, 19 Jun 2018 14:08:44 +0200
> 
> I tried the patched follow.el (the one that contains the modified
> follow-calc-win-end) on my somewhat slower (but also new quad core)
> office machine, unfortunately no amelioration of lag was noticeable.
> 
> Same procedure: patching follow.el, starting the profiler, loading the
> patched uncompiled version into emacs via M-x load-file and finally
> start editing.
> 
> The lag remains enormous (e.g. more than a second lag between typing RET
> and seeing a newline on screen); I noticed however, that upon typing RET
> one time, the newline comes almost immediately; but after typing more
> RETs they take ages to appear on screen.
> 
> Bizarrely, this happened also: redisplay takes very long and it
> apparently aborts leaving paragraphs before point unfilled, one long
> line of text (I always use auto-fill-mode).

Then this method of debugging the problem is no longer efficient, I'd
need to run this on my system to see what's going on.

So please use "M-x report-emacs-bug RET" to file a bug report about
this, and please attach to the report the Org file where you observe
the problems.  (You can replace the text of the headings and the
bodies of your entries with some random character, to protect your
private stuff, but please leave the parts that control Org mode
intact.)

Thanks.



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-16 10:25 follow-mode: extremely slow in combination with org-mode Gerald Wildgruber
  2018-06-16 11:41 ` Eli Zaretskii
@ 2018-06-21  8:25 ` Anders Lindgren
  2018-06-27 16:43   ` Gerald Wildgruber
  1 sibling, 1 reply; 18+ messages in thread
From: Anders Lindgren @ 2018-06-21  8:25 UTC (permalink / raw)
  To: Gerald Wildgruber; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 5178 bytes --]

 Hi!

There can be a number of reasons why follow-mode is slow. The main problem
is that is it's time consuming to recompute the window state, and even more
time consuming when aligning the windows. For simple commands like point
movement, follow-mode assumes that the content of the buffers hasn't
changed and retain the existing layout, whereas for other commands
(including self-insert-command), everything is recomputed and realigned.

However, I use six side by side windows, and a small font, giving me a
total of almost 800 consecutive lines, and it typically runs fine without
any noticeable lag.

There are a number of factors that can have a negative impact on
follow-mode:

- It run slower on a 32 bit binary than on a 64 bit binary (at least under
Windows). (For this reason, I used Emacs 22 at work for many years, until
prebuilt 64 bit binaries of modern Emacs versions were available.)

- Follow-mode is has problems when the width of the windows differ,
especially if you have long lines that spill over from one window to
another. I've written a support package to set up side-by-side windows in a
pixel-perfect way, https://github.com/Lindydancer/multicolumn -- please try
if and see if follow-mode runs more smoothly.

One thing that I have had on my wish-list for a very long time (20+ years)
is that normal typing should use and update the cache, to speed things up.
Of course, this only work as long as the editing doesn't change the layout,
like when inserting a newline or make a line so long that it wraps.
Unfortunately, I don't think that I will have time for it anytime soon, but
I'm happy to share my ideas if someone else is willing to make a try.

What made me curious that you said that the slow-down only occurs with
org-mode. My guess is that invisible text makes it harder for Emacs to
calculate window layout properties.

Sincerely,
    Anders Lindgren (I wrote follow-mode when I was a student, in the mid
1990:s)


On Sat, Jun 16, 2018 at 12:25 PM, Gerald Wildgruber <wildgruber@tu-berlin.de
> wrote:

>
> Hi,
>
> I have got a problem with enormous lag while entering text in buffers
> with emacs follow-mode enabled.
>
> I'm using emacs (git checkout v. 27.0.50) and Org mode (git checkout
> release_9.1.13-760-g8def68).
>
> My work is solely text-based, using org-mode. A typical setup is to use
> a maximized or full screen emacs frame split into five windows
> positioned vertically one next to the other on a 40" 4k display. All
> five windows display one and the same file, that is opened using emacs
> follow-mode, so that every window is displaying a portion of the same
> file in a continuous manner (simultaneously displaying up to 45kb of
> text in one frame, an entire paper).
>
> Unfortunately, with this setup there is terrible lag with every single
> key input (on a very fast quad core machine); every key stroke produces
> a 100% processor load.
>
> If I deactivate follow-mode, the problem disappears. It also gets better
> if I enable text-mode instead of org-mode.
>
> I then used the elisp profiler (M-x profiler-start/report) to find out
> which function uses most cpu time while editing text in said setup. Here
> is the result:
>
> Collapsed, the report looks like that:
>
> + command-execute                                                8789  47%
> + follow-post-command-hook                                       7755  41%
> + ...                                                            1976  10%
> + redisplay_internal (C function)                                 104   0%
> + yas--post-command-handler                                        40   0%
> + timer-event-handler                                              20   0%
>   tooltip-hide                                                      7   0%
>
>
> And somewhat expanded:
>
> - follow-post-command-hook                                       7755  41%
>  - follow-adjust-window                                          7755  41%
>   - follow-windows-start-end                                     7732  41%
>    - follow-calc-win-end                                         7732  41%
>     + pos-visible-in-window-p                                      25   0%
>     + posn-at-x-y                                                   7   0%
>     + window-inside-pixel-edges                                     3   0%
>   + follow-all-followers                                            4   0%
>     follow-avoid-tail-recenter                                      3   0%
>
> If I understand correctly "follow-calc-win-end" would be the function
> that uses most of cpu time.
>
> I then did additional profiling with the elp library, adding relevant
> functions under "Command-execute" and "follow-post-command-hook" to
> elp-function-list, edited text, and then did M-x elp-results. It showed
> again that "follow-calc-win-end" by far used most of cpu time.
>
> Anyone got an idea what's going on here and how to debug that? Are there
> possible optimizations with this situation? Or is this "normal",
> expected behavior, simply due to the number of windows and text
> displayed?
>
> Thanks
>
> Gerald.
>
> ---------------------
> Sent with mu4e
>
>

[-- Attachment #2: Type: text/html, Size: 8166 bytes --]

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: follow-mode: extremely slow in combination with org-mode
  2018-06-21  8:25 ` Anders Lindgren
@ 2018-06-27 16:43   ` Gerald Wildgruber
  2018-07-09 20:12     ` Anders Lindgren
  0 siblings, 1 reply; 18+ messages in thread
From: Gerald Wildgruber @ 2018-06-27 16:43 UTC (permalink / raw)
  To: Anders Lindgren; +Cc: emacs-devel


Hi Anders

thanks for your reply: I just returned to office and couldn't answer
earlier!

On Do, Jun 21 2018, Anders Lindgren <andlind@gmail.com> wrote:

>  Hi!
>
> There can be a number of reasons why follow-mode is slow. The main problem
> is that is it's time consuming to recompute the window state, and even more
> time consuming when aligning the windows. For simple commands like point
> movement, follow-mode assumes that the content of the buffers hasn't
> changed and retain the existing layout, whereas for other commands
> (including self-insert-command), everything is recomputed and realigned.
>
> However, I use six side by side windows, and a small font, giving me a
> total of almost 800 consecutive lines, and it typically runs fine without
> any noticeable lag.

That's quite close to my setup.

However: when I use org-mode, with entire branches collapsed, there
might well be several thousand (2000-15000) lines of text in the buffer;
if all this is recalculated every time I enter a letter, -- perhaps the
lags I observe are "normal". But then, the machine I'm working on is a
really fast and new computer.

> There are a number of factors that can have a negative impact on
> follow-mode:

> - It run slower on a 32 bit binary than on a 64 bit binary (at least under
> Windows). (For this reason, I used Emacs 22 at work for many years, until
> prebuilt 64 bit binaries of modern Emacs versions were available.)

Mine are 64 bit machines.

> - Follow-mode is has problems when the width of the windows differ,
> especially if you have long lines that spill over from one window to
> another. I've written a support package to set up side-by-side windows in a
> pixel-perfect way, https://github.com/Lindydancer/multicolumn -- please try
> if and see if follow-mode runs more smoothly.

Actually I use this already! I have in my .emacs the following:

(require 'multicolumn)
(multicolumn-global-mode 1)

> One thing that I have had on my wish-list for a very long time (20+ years)
> is that normal typing should use and update the cache, to speed things up.
> Of course, this only work as long as the editing doesn't change the layout,
> like when inserting a newline or make a line so long that it wraps.
> Unfortunately, I don't think that I will have time for it anytime soon, but
> I'm happy to share my ideas if someone else is willing to make a try.

I hope someone picks this up!!! :-)

Can you tell if the function that uses so much of cpu time --
follow-calc-win-end -- in the form it is now part of new emacs versions,
has any evident problem compared to your own longer version in pre
version 22 emacs? Cf Alan's posting in this thread:
https://lists.gnu.org/archive/html/emacs-devel/2018-06/msg00522.html

> What made me curious that you said that the slow-down only occurs with
> org-mode. My guess is that invisible text makes it harder for Emacs to
> calculate window layout properties.

Yes: the lag is much lower with text-mode.

Thanks again

Gerald.

> Sincerely,
>     Anders Lindgren (I wrote follow-mode when I was a student, in the mid
> 1990:s)
>
>
> On Sat, Jun 16, 2018 at 12:25 PM, Gerald Wildgruber <wildgruber@tu-berlin.de
>> wrote:
>
>>
>> Hi,
>>
>> I have got a problem with enormous lag while entering text in buffers
>> with emacs follow-mode enabled.
>>
>> I'm using emacs (git checkout v. 27.0.50) and Org mode (git checkout
>> release_9.1.13-760-g8def68).
>>
>> My work is solely text-based, using org-mode. A typical setup is to use
>> a maximized or full screen emacs frame split into five windows
>> positioned vertically one next to the other on a 40" 4k display. All
>> five windows display one and the same file, that is opened using emacs
>> follow-mode, so that every window is displaying a portion of the same
>> file in a continuous manner (simultaneously displaying up to 45kb of
>> text in one frame, an entire paper).
>>
>> Unfortunately, with this setup there is terrible lag with every single
>> key input (on a very fast quad core machine); every key stroke produces
>> a 100% processor load.
>>
>> If I deactivate follow-mode, the problem disappears. It also gets better
>> if I enable text-mode instead of org-mode.
>>
>> I then used the elisp profiler (M-x profiler-start/report) to find out
>> which function uses most cpu time while editing text in said setup. Here
>> is the result:
>>
>> Collapsed, the report looks like that:
>>
>> + command-execute                                                8789  47%
>> + follow-post-command-hook                                       7755  41%
>> + ...                                                            1976  10%
>> + redisplay_internal (C function)                                 104   0%
>> + yas--post-command-handler                                        40   0%
>> + timer-event-handler                                              20   0%
>>   tooltip-hide                                                      7   0%
>>
>>
>> And somewhat expanded:
>>
>> - follow-post-command-hook                                       7755  41%
>>  - follow-adjust-window                                          7755  41%
>>   - follow-windows-start-end                                     7732  41%
>>    - follow-calc-win-end                                         7732  41%
>>     + pos-visible-in-window-p                                      25   0%
>>     + posn-at-x-y                                                   7   0%
>>     + window-inside-pixel-edges                                     3   0%
>>   + follow-all-followers                                            4   0%
>>     follow-avoid-tail-recenter                                      3   0%
>>
>> If I understand correctly "follow-calc-win-end" would be the function
>> that uses most of cpu time.
>>
>> I then did additional profiling with the elp library, adding relevant
>> functions under "Command-execute" and "follow-post-command-hook" to
>> elp-function-list, edited text, and then did M-x elp-results. It showed
>> again that "follow-calc-win-end" by far used most of cpu time.
>>
>> Anyone got an idea what's going on here and how to debug that? Are there
>> possible optimizations with this situation? Or is this "normal",
>> expected behavior, simply due to the number of windows and text
>> displayed?
>>
>> Thanks
>>
>> Gerald.
>>
>> ---------------------
>> Sent with mu4e
>>
>>


--
Dr. Gerald Wildgruber
Institut für Philosophie, Literatur-, Wissenschafts- und Technikgeschichte
Literaturwissenschaft mit Schwerpunkt Literatur und Wissenschaft
Technische Universität Berlin
Straße des 17. Juni 135
D-10623 Berlin
http://www.philosophie.tu-berlin.de/menue/home/
T. +49 (0)30 314 25924
F. +49 (0)30 314 23107
wildgruber@tu-berlin.de
---------------------
Sent with mu4e



^ permalink raw reply	[flat|nested] 18+ messages in thread

* follow-mode: extremely slow in combination with org-mode
  2018-06-27 16:43   ` Gerald Wildgruber
@ 2018-07-09 20:12     ` Anders Lindgren
  0 siblings, 0 replies; 18+ messages in thread
From: Anders Lindgren @ 2018-07-09 20:12 UTC (permalink / raw)
  To: Gerald Wildgruber; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 3621 bytes --]

Hi!

I decided to go to the bottom of this, and had to do some digging, which
took a little bit longer than I thought it would do, sorry about the delay.


On Wed, Jun 27, 2018 at 6:43 PM, Gerald Wildgruber <wildgruber@tu-berlin.de>
wrote:

> > - Follow-mode is has problems when the width of the windows differ,
> > especially if you have long lines that spill over from one window to
> > another. I've written a support package to set up side-by-side windows
> in a
> > pixel-perfect way, https://github.com/Lindydancer/multicolumn -- please
> try
> > if and see if follow-mode runs more smoothly.
>
> Actually I use this already! [...]
>

You just made my day!


Can you tell if the function that uses so much of cpu time --
> follow-calc-win-end -- in the form it is now part of new emacs versions,
> has any evident problem compared to your own longer version in pre
> version 22 emacs? Cf Alan's posting in this thread:
> https://lists.gnu.org/archive/html/emacs-devel/2018-06/msg00522.html
>
>
The main difference is that the current version handles clipped lines at
the end of a window. My original version predates the addition of the
UPDATE optional parameter to window-end. Effectively, it calculated
window-end plus one. Anyway, it would not work in a modern Emacs where
fonts of different heights could be used.

The modern version handles clipped lines by repeating a clipped screen line
in the next window. This is for two reasons:

1) Lines where only one or a few pixels are displayed would effectively be
hidden from users, if they weren't repeated in the next window.

2) When the cursor is placed on a clipped line, Emacs recenters the window,
which breaks Follow mode. Instead, Follow mode selects the next window,
which is displaying the same line. (Or rather, this is the way it was
supposed to work, I just noticed that it is broken in Emacs 26.1 -- the
original window is scrolled anyway. I don't know how long it as been
broken.)

Lines in this context are screen lines, not physical lines. In other words,
if you have a long wrapped physical line, the function should return a
suitable position inside the line where the next window should start.

The follow-calc-win-end function should return window-end plus one if the
line isn't clipped, and the beginning of the bottommost screen line if it
is clipped. (The patch supplied by Eli doesn't do this, so it won't work --
but I guess its main purpose of the patch was to figure out which parts of
the function consume processing power.)

So, where do we go from here? First, I noticed that pos-visible-in-window-p
can take an argument PARTIALLY that, when non-nil, makes the function
provide information about whether or not the position is partially visible.
Presumably, it could replace the pixel juggling the current function do.
However, if the big time thief is `window-end` with an non-nil UPDATE
argument we must either make is faster (at least after self-insert-command)
or figure out a way to avoid calling it too often, somehow.

Being (somewhat) older and ((hopefully) somewhat) wiser, I wish I had
written a test suite for Follow mode, or at least a test protocol. However,
Follow mode was written 12 years before work on Ert even had started...
Anyway, having a test suite containing all the corner cases would make it
easier to try out alternative implementations, and it would prevent us from
introducing bugs like the one I just found.

I wished I had more time to work on this, but it took me a week just to
find the time to write this mail, so clearly I'm not the best candidate at
the moment.

Sincerely,
    Anders Lindgren

[-- Attachment #2: Type: text/html, Size: 4915 bytes --]

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2018-07-09 20:12 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-16 10:25 follow-mode: extremely slow in combination with org-mode Gerald Wildgruber
2018-06-16 11:41 ` Eli Zaretskii
2018-06-16 13:06   ` Gerald Wildgruber
2018-06-16 16:25     ` Eli Zaretskii
2018-06-16 21:05       ` Gerald Wildgruber
2018-06-17  6:57       ` Gerald Wildgruber
2018-06-17  8:12         ` Eli Zaretskii
2018-06-17 11:13           ` Alan Mackenzie
2018-06-17 13:02             ` Eli Zaretskii
2018-06-17 14:56               ` Gerald Wildgruber
2018-06-17 15:10                 ` Eli Zaretskii
2018-06-19 12:08                   ` Gerald Wildgruber
2018-06-19 16:55                     ` Eli Zaretskii
2018-06-19 13:53                   ` Gerald Wildgruber
2018-06-17 15:13             ` Eli Zaretskii
2018-06-21  8:25 ` Anders Lindgren
2018-06-27 16:43   ` Gerald Wildgruber
2018-07-09 20:12     ` Anders Lindgren

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).