Org mode in combination with emacs follow-mode is terrible

Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

Hi,

I'm using emacs (git checkout v. 27.0.50) and Org mode (git checkout
release_9.1.13-760-g8def68).

My typical setup is to use a maximized or full screen emacs frame split
into five windows positioned vertically one next to the other. All five
windows display one and the same file, that is opened using emacs
follow-mode, so that every window is displaying a portion of the same
file in a continuous manner.

Yet, there seems to be an incompatibility between Org mode and
follow-mode: there is terrible lag with every single key input (on a
very fast quad core machine); every key stroke produces a 100% processor
load. It would seem that this is even worse, when working on portions
towards the end of the file. If I deactivate follow-mode, the problem
disappears.

Anyone got an idea what's going on here and how to debug that?

Thanks

Gerald.

---------------------
Sent with mu4e

Re: Org mode in combination with emacs follow-mode is terrible

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On Monday, 11 Jun 2018 at 14:56, Gerald Wildgruber wrote:
> My typical setup is to use a maximized or full screen emacs frame split

[...]

> portions towards the end of the file. If I deactivate follow-mode, the
> problem disappears.

I have a 38" monitor and I frequently do what you say.  However, I
typically only split the frame into 3 windows.  I don't remember seeing
any lag.  But I am currently away so cannot verify.  What happens if you
try with less windows?

-- 
Eric S Fraga via Emacs 25.2.2, Org release_9.1.13-791-g842002

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

Hi Eric,

thanks for the input: indeed, I always use at least five windows (on a
40" display), as this setup displays simultanously almost 60Kb of text,
an entire papaer, which is excellent!

I will try the results with fewer windows tomorrow, when back in office.

Elsewise: do you see any reason or issue with both modes enabled that
could lead to such behavior: it would seem that emacs tries to calculate
something with every new keystroke, that involves all text before point.

Gerald.

On Di, Jun 12 2018, Eric S Fraga <esflists@gmail.com> wrote:

> On Monday, 11 Jun 2018 at 14:56, Gerald Wildgruber wrote:
>> My typical setup is to use a maximized or full screen emacs frame split
>
> [...]
>
>> portions towards the end of the file. If I deactivate follow-mode, the
>> problem disappears.
>
> I have a 38" monitor and I frequently do what you say.  However, I
> typically only split the frame into 3 windows.  I don't remember seeing
> any lag.  But I am currently away so cannot verify.  What happens if you
> try with less windows?


--
Dr. Gerald Wildgruber
Institut für Philosophie, Literatur-, Wissenschafts- und Technikgeschichte
Literaturwissenschaft mit Schwerpunkt Literatur und Wissenschaft
Technische Universität Berlin
Straße des 17. Juni 135
D-10623 Berlin
http://www.philosophie.tu-berlin.de/menue/home/
T. +49 (0)30 314 25924
F. +49 (0)30 314 23107
wildgruber@tu-berlin.de
---------------------
Sent with mu4e

Re: Org mode in combination with emacs follow-mode is terrible

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

On Tuesday, 12 Jun 2018 at 12:23, Gerald Wildgruber wrote:
> Hi Eric,
>
> thanks for the input: indeed, I always use at least five windows (on a
> 40" display), as this setup displays simultanously almost 60Kb of text,
> an entire papaer, which is excellent!

That's a lot of text so maybe it's really not surprising maybe that
there is a lag.  What happens if you edit the same file in text-mode?

> Elsewise: do you see any reason or issue with both modes enabled that
> could lead to such behavior: it would seem that emacs tries to
> calculate something with every new keystroke, that involves all text
> before point.

Maybe font-lock and org-indent may impact on this?

I also find that the presence of tables often slows org down although
it's difficult to quantify the actual impact.

-- 
Eric S Fraga via Emacs 25.2.2, Org release_9.1.13-791-g842002

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

Thanks! actually every single one of the following does help a bit:

- reducing number of windows to 3
- switching to text-mode
- dectivating org-indent-mode, with org-mode still being active

There's still lag, but not as extreme as with my original setup.

Still, the most significant change occurs by deactivating follow-mode:
every lag simply disappears.

Gerald.

On Di, Jun 12 2018, Eric S Fraga <esflists@gmail.com> wrote:

> On Tuesday, 12 Jun 2018 at 12:23, Gerald Wildgruber wrote:
>> Hi Eric,
>>
>> thanks for the input: indeed, I always use at least five windows (on a
>> 40" display), as this setup displays simultanously almost 60Kb of text,
>> an entire papaer, which is excellent!
>
> That's a lot of text so maybe it's really not surprising maybe that
> there is a lag.  What happens if you edit the same file in text-mode?
>
>> Elsewise: do you see any reason or issue with both modes enabled that
>> could lead to such behavior: it would seem that emacs tries to
>> calculate something with every new keystroke, that involves all text
>> before point.
>
> Maybe font-lock and org-indent may impact on this?
>
> I also find that the presence of tables often slows org down although
> it's difficult to quantify the actual impact.


--
Dr. Gerald Wildgruber
Institut für Philosophie, Literatur-, Wissenschafts- und Technikgeschichte
Literaturwissenschaft mit Schwerpunkt Literatur und Wissenschaft
Technische Universität Berlin
Straße des 17. Juni 135
D-10623 Berlin
http://www.philosophie.tu-berlin.de/menue/home/
T. +49 (0)30 314 25924
F. +49 (0)30 314 23107
wildgruber@tu-berlin.de
---------------------
Sent with mu4e

Re: Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

Switching to text-mode, with 5 windows and follow-mode still being
active reduces lag significantly.

So there must be an issue specifically with the combination of org-mode
and follow-mode!

Gerald.

On Di, Jun 12 2018, Eric S Fraga <esflists@gmail.com> wrote:

> On Tuesday, 12 Jun 2018 at 12:23, Gerald Wildgruber wrote:
>> Hi Eric,
>>
>> thanks for the input: indeed, I always use at least five windows (on a
>> 40" display), as this setup displays simultanously almost 60Kb of text,
>> an entire papaer, which is excellent!
>
> That's a lot of text so maybe it's really not surprising maybe that
> there is a lag.  What happens if you edit the same file in text-mode?
>
>> Elsewise: do you see any reason or issue with both modes enabled that
>> could lead to such behavior: it would seem that emacs tries to
>> calculate something with every new keystroke, that involves all text
>> before point.
>
> Maybe font-lock and org-indent may impact on this?
>
> I also find that the presence of tables often slows org down although
> it's difficult to quantify the actual impact.


--
Dr. Gerald Wildgruber
Institut für Philosophie, Literatur-, Wissenschafts- und Technikgeschichte
Literaturwissenschaft mit Schwerpunkt Literatur und Wissenschaft
Technische Universität Berlin
Straße des 17. Juni 135
D-10623 Berlin
http://www.philosophie.tu-berlin.de/menue/home/
T. +49 (0)30 314 25924
F. +49 (0)30 314 23107
wildgruber@tu-berlin.de
---------------------
Sent with mu4e

Re: Org mode in combination with emacs follow-mode is terrible

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 933 bytes --]

On Wednesday, 13 Jun 2018 at 09:53, Gerald Wildgruber wrote:
> Switching to text-mode, with 5 windows and follow-mode still being
> active reduces lag significantly.
>
> So there must be an issue specifically with the combination of org-mode
> and follow-mode!

I don't think there's an issue per se in the sense of bugs.  Org does
much more processing of the text than does text mode so if you have 60k
worth of text to process each time you type something, it's probably not
surprising that there is a lag.  60k is a lot of text and I assume there
are many headings etc.?  Part of the attraction of org mode is the
visual representation of the text and this unfortunately comes at a
cost.

I cannot remember what version of Emacs you are using.  If not the
latest, it might be worth trying v26.1 or even one of the latest
snapshots in case?


-- 
Eric S Fraga via Emacs 25.2.2, Org release_9.1.13-791-g842002

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: Org mode in combination with emacs follow-mode is terrible

[Joost Kremers]

On Wed, Jun 13 2018, Eric S Fraga wrote:
> On Wednesday, 13 Jun 2018 at 09:53, Gerald Wildgruber wrote:
>> Switching to text-mode, with 5 windows and follow-mode still 
>> being
>> active reduces lag significantly.
>>
>> So there must be an issue specifically with the combination of 
>> org-mode
>> and follow-mode!
>
> I don't think there's an issue per se in the sense of bugs.  Org 
> does
> much more processing of the text than does text mode so if you 
> have 60k
> worth of text to process each time you type something, it's 
> probably not
> surprising that there is a lag.

Actually, I would suspect it's more of a problem for follow-mode 
than org-mode, because follow-mode needs to keep the different 
windows in sync. For this, it adds a function to 
`post-command-hook', which means it's run after every key press.

It's not inconceivable that `follow-mode' does something that is 
extra time-consuming in an Org buffer. To find out what that might 
be, you could try the Elisp profiler that comes with Emacs. See 
the section "Profiling" in the Elisp manual for details.

Once you've found out which function(s) consume so much time, it 
might be possible to ask in here or on emacs-devel what exactly is 
causing the problem and whether there's a way around it.

-- 
Joost Kremers
Life has its moments

Re: Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

Thanks Joost,

that's interesting! I never used the profiler but I will try and see how
far I get with it and then come back here.

Gerald.

On Mi, Jun 13 2018, Joost Kremers <joostkremers@fastmail.fm> wrote:

> On Wed, Jun 13 2018, Eric S Fraga wrote:
>> On Wednesday, 13 Jun 2018 at 09:53, Gerald Wildgruber wrote:
>>> Switching to text-mode, with 5 windows and follow-mode still being
>>> active reduces lag significantly.
>>>
>>> So there must be an issue specifically with the combination of org-mode
>>> and follow-mode!
>>
>> I don't think there's an issue per se in the sense of bugs.  Org does
>> much more processing of the text than does text mode so if you have 60k
>> worth of text to process each time you type something, it's probably not
>> surprising that there is a lag.
>
> Actually, I would suspect it's more of a problem for follow-mode than
> org-mode, because follow-mode needs to keep the different windows in sync.
> For this, it adds a function to `post-command-hook', which means it's run
> after every key press.
>
> It's not inconceivable that `follow-mode' does something that is extra
> time-consuming in an Org buffer. To find out what that might be, you could
> try the Elisp profiler that comes with Emacs. See the section "Profiling"
> in the Elisp manual for details.
>
> Once you've found out which function(s) consume so much time, it might be
> possible to ask in here or on emacs-devel what exactly is causing the
> problem and whether there's a way around it.


--
Dr. Gerald Wildgruber
Institut für Philosophie, Literatur-, Wissenschafts- und Technikgeschichte
Literaturwissenschaft mit Schwerpunkt Literatur und Wissenschaft
Technische Universität Berlin
Straße des 17. Juni 135
D-10623 Berlin
http://www.philosophie.tu-berlin.de/menue/home/
T. +49 (0)30 314 25924
F. +49 (0)30 314 23107
wildgruber@tu-berlin.de
---------------------
Sent with mu4e

Re: Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

I tried profiling my use of org-mode in conjunction with follow-mode; I
did the following:

1. M-x profiler-start
2. I enabled follow-mode in my buffer
3. I did some editing
4. M-x profiler-report
5. M-x profiler-stop

here's the result:

Collapsed, the report looks like that:

+ command-execute                                                8789  47%
+ follow-post-command-hook                                       7755  41%
+ ...                                                            1976  10%
+ redisplay_internal (C function)                                 104   0%
+ yas--post-command-handler                                        40   0%
+ timer-event-handler                                              20   0%
  tooltip-hide                                                      7   0%


And somewhat expanded:

- command-execute                                                8789  47%
 - call-interactively                                            8789  47%
  - byte-code                                                    8174  43%
   - read-extended-command                                       8174  43%
    - completing-read                                            8174  43%
     - #<compiled 0x3d2ceb9>                                     8174  43%
      - apply                                                    8174  43%
       - helm--completing-read-default                           8174  43%
        - helm-completing-read-sync-default-handler               8174  43%
         - helm-completing-read-default-1                        8174  43%
          - helm-comp-read                                       8174  43%
           - helm                                                8165  43%
            - apply                                              8165  43%
             - helm                                              8165  43%
              - apply                                            8165  43%
               - helm-internal                                   8165  43%
                - helm-read-pattern-maybe                        8150  43%
                 + read-from-minibuffer                          8079  43%
                 + helm-update                                     43   0%
                - #<compiled 0x12071ed>                             9   0%
                 + helm-cleanup                                     9   0%
                + helm-display-buffer                               6   0%
           + helm-make-source                                       9   0%
  + funcall-interactively                                         615   3%
- follow-post-command-hook                                       7755  41%
 - follow-adjust-window                                          7755  41%
  - follow-windows-start-end                                     7732  41%
   - follow-calc-win-end                                         7732  41%
    + pos-visible-in-window-p                                      25   0%
    + posn-at-x-y                                                   7   0%
    + window-inside-pixel-edges                                     3   0%
  + follow-all-followers                                            4   0%
    follow-avoid-tail-recenter                                      3   0%

If I understand correctly "follow-calc-win-end" would be the function
that uses most of cpu time.

I can't say how "normal" the other 47% under "command-execute" are,
which mainly seems to be helm-related ("helm-update").

Can anyone give an advice on possible optimizations with this situation?


Gerald.


On Mi, Jun 13 2018, Joost Kremers <joostkremers@fastmail.fm> wrote:

> Actually, I would suspect it's more of a problem for follow-mode than
> org-mode, because follow-mode needs to keep the different windows in sync.
> For this, it adds a function to `post-command-hook', which means it's run
> after every key press.
>
> It's not inconceivable that `follow-mode' does something that is extra
> time-consuming in an Org buffer. To find out what that might be, you could
> try the Elisp profiler that comes with Emacs. See the section "Profiling"
> in the Elisp manual for details.
>
> Once you've found out which function(s) consume so much time, it might be
> possible to ask in here or on emacs-devel what exactly is causing the
> problem and whether there's a way around it.


--
Dr. Gerald Wildgruber
Institut für Philosophie, Literatur-, Wissenschafts- und Technikgeschichte
Literaturwissenschaft mit Schwerpunkt Literatur und Wissenschaft
Technische Universität Berlin
Straße des 17. Juni 135
D-10623 Berlin
http://www.philosophie.tu-berlin.de/menue/home/
T. +49 (0)30 314 25924
F. +49 (0)30 314 23107
wildgruber@tu-berlin.de
---------------------
Sent with mu4e

Re: Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

Thanks again; I'm always using the latest git checkout of emacs
(v.27, see my first posting).

On Mi, Jun 13 2018, Eric S Fraga <esflists@gmail.com> wrote:

> I don't think there's an issue per se in the sense of bugs.  Org does
> much more processing of the text than does text mode so if you have 60k
> worth of text to process each time you type something, it's probably not
> surprising that there is a lag.  60k is a lot of text and I assume there
> are many headings etc.?

Yes, indeed, there are! Actually does this mean that emacs or org-mode
has to take into consideration every single bit of text in the visible
part buffer of the buffer, whenever I press a key? what needs to be
calculated by simple entering of text?

Gerald

--
Dr. Gerald Wildgruber
Institut für Philosophie, Literatur-, Wissenschafts- und Technikgeschichte
Literaturwissenschaft mit Schwerpunkt Literatur und Wissenschaft
Technische Universität Berlin
Straße des 17. Juni 135
D-10623 Berlin
http://www.philosophie.tu-berlin.de/menue/home/
T. +49 (0)30 314 25924
F. +49 (0)30 314 23107
wildgruber@tu-berlin.de
---------------------
Sent with mu4e

Org mode in combination with emacs follow-mode is terrible

[Gerald Wildgruber]

Hi,

I'm using emacs (git checkout v. 27.0.50) and Org mode (git checkout
release_9.1.13-760-g8def68).

My typical setup is to use a maximized or full screen emacs frame split
into five windows positioned vertically one next to the other. All five
windows display one and the same file, that is opened using emacs
follow-mode, so that every window is displaying a portion of the same
file in a continuous manner.

Yet, there seems to be an incompatibility between Org mode and
follow-mode: there is terrible lag with every single key input (on a
very fast quad core machine); every key stroke produces a 100% processor
load. It would seem that this is even worse, when working on portions
towards the end of the file. If I deactivate follow-mode, the problem
disappears.

Anyone got an idea what's going on here and how to debug that?

Thanks

Gerald.

---------------------
Sent with mu4e