From mboxrd@z Thu Jan 1 00:00:00 1970 From: JI Xiang Subject: Re: Ridiculously long (2 minutes) startup time for org agenda Date: Mon, 12 Mar 2018 13:29:36 +0100 Message-ID: <06da26bc-5e4c-e7f6-a2df-51121a285a6f@gmail.com> References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="------------D7698C964DD686204A49B04A" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:36419) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1evMaF-0003aa-Hb for emacs-orgmode@gnu.org; Mon, 12 Mar 2018 08:29:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1evMaC-0006NF-Ak for emacs-orgmode@gnu.org; Mon, 12 Mar 2018 08:29:43 -0400 Received: from mail-wr0-x232.google.com ([2a00:1450:400c:c0c::232]:33405) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1evMaB-0006Kw-SM for emacs-orgmode@gnu.org; Mon, 12 Mar 2018 08:29:40 -0400 Received: by mail-wr0-x232.google.com with SMTP id r8so329184wrg.0 for ; Mon, 12 Mar 2018 05:29:39 -0700 (PDT) In-Reply-To: Content-Language: en-US List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: Daniel Clemente Cc: Org-mode Org-Mode This is a multi-part message in MIME format. --------------D7698C964DD686204A49B04A Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit I finally found out the problem. I've been using Spacemacs and the `spell-checking` layer would automatically initiate flyspell on each org file, resulting in tremendous delay. If I disable flyspell from being automatically activated, the agenda is shown in a few seconds. On 26.02.2018 17:29, Daniel Clemente wrote: > It can be many things. My emacs was taking 6 minutes to open for some > years, due to many reasons: a slow computer, many org files and very > large (>100 files, >25 Mb in total), lack of optimizations from my > side and from org-mode's side. I got used to it but it's a very bad > experience, specially when it crashes (and I made it crash a lot). > Now org-mode has improved and I disabled the slow parts, and it opens > very fast (<20 seconds). I still open large files. > > The problem may not only be in org but in emacs (e.g. vc-mode can be > slow), though from the instrumentation you send, it doesn't seem to be > the case because it doesn't spend time opening the files. > > org-agenda-prepare-buffers is the slow function, so it's the one you > should check. Try to use edebug in it (C-u M-C-x) and run it slowly to > see in which section it's the slowest. Then optimize that part. You'll > see many optimization there (ignore properties, etc.). I had to > disable many slow things, like <<>> (when you use a lot, e.g. > 1000 per file, it's too slow and even crashes), I think that these > radio targets were the main reason of the slowness. > > Try creating a very large file and then check opening it, to see if > it's also slow. > Try things in a clean emacs (e.g. starting with: emacs -Q), because > you may have some other mode enabled that wants to do lots of things > when you open the files. > > You can test many more things. But keep testing, because it's possible > to make it work faster. > > > On Mon, Feb 19, 2018 at 3:00 AM, JI Xiang > wrote: > > Hello, first time posting on the mailing list. I encountered a > problem where my agenda view (|C-c a a|) is taking ridiculously > long (nearly 2 minutes) to be shown the first time. From the > second time onwards, the time is about 11 seconds, which is still > very long by any means. > > I wondered whether it has something to do with me having many > files in the agenda list, so I tried to restrict the list to just > one file. But still the first call to org-agenda-list took 12 > seconds. This shouldn’t be normal, right? > > The following are shown in the |Messages| buffer during the call. > > |Press key for agenda command: Restoring clock data Loading > /home/jx/.emacs.d/.cache/org-clock-save.el (source)...done [yas] > Prepared just-in-time loading of snippets successfully. > Importmagic and/or epc not found. importmagic.el will not be > working. Setting up indent for shell type zsh Indentation > variables are now local. Indentation setup for shell type zsh > Using vacuous schema Shell native completion is disabled, using > fallback | > > I’m not sure why they would be there, especially the “using > vacuous schema” and “shell native completion” part. Are the > messages related to some |#BEGIN_SRC| blocks in the org files? I > don’t think I would ever need source code blocks when I’m viewing > an agenda buffer? > > I posted the question on Emacs.SE > > and somebody suggested me use |elp-instrument| to perform a > profiling. The results are as follows: > > |Function Name, Call Count, Elapsed Time, Average Time org-agenda 1 > 116.6048159 116.6048159 org-agenda-list 1 116.29427357 > 116.29427357 org-agenda-prepare 1 109.15345470 109.15345470 > org-agenda-prepare-buffers 1 108.98258905 108.98258905 > org-agenda-get-day-entries 1288 7.0089191339 0.0054417074 > org-agenda-get-scheduled 1288 3.726361062 0.0028931374 > org-agenda-get-deadlines 1288 2.1579713230 0.0016754435 > org-agenda--timestamp-to-absolute 14544 1.1317418120 7.781...e-05 > org-agenda-get-timestamps 1288 0.3673404320 0.0002852021 > org-agenda-get-sexps 1288 0.3438970410 0.0002670008 > org-agenda-get-restriction-and-command 1 0.310503975 0.310503975 > org-agenda-get-blocks 1288 0.3083237900 0.0002393818 > org-agenda-prepare-window 1 0.157091024 0.157091024 > org-agenda-skip 8624 0.0781296389 9.059...e-06 org-agenda-files 49 > 0.026090686 0.0005324629 org-agenda-finalize 1 0.013325178 > 0.013325178 org-agenda-mode 1 0.009848546 0.009848546 > org-agenda-finalize-entries 23 0.006178961 0.0002686504 > org-agenda-today-p 2604 0.0061717839 2.370...e-06 > org-agenda-skip-eval 16968 0.0052723920 3.107...e-07 > org-agenda-highlight-todo 112 0.0039840550 3.557...e-05 > org-agenda-format-item 112 0.0038309749 3.420...e-05 > org-agenda-new-marker 208 0.0031529359 1.515...e-05 > org-agenda-format-date-aligned 28 0.0007373120 2.633...e-05 > org-agenda-add-time-grid-maybe 28 0.0003810809 1.361...e-05 > org-agenda-fix-displayed-tags 112 0.000342384 3.057e-06 > org-agenda-fontify-priorities 1 0.00018465 0.00018465 > org-agenda-align-tags 1 0.000160119 0.000160119 > org-agenda-get-day-face 28 0.000156385 5.585...e-06 > org-agenda-get-category-icon 112 7.702...e-05 6.877...e-07 > org-agenda-span-name 23 4.4484e-05 1.934...e-06 > org-agenda-span-to-ndays 1 2.3496e-05 2.3496e-05 > org-agenda-fit-window-to-buffer 1 2.0877e-05 2.0877e-05 > org-agenda-time-of-day-to-ampm-maybe 21 1.124...e-05 5.353...e-07 > org-agenda-set-mode-name 1 9.226e-06 9.226e-06 > org-agenda-mark-header-line 1 8.774e-06 8.774e-06 > org-agenda-reset-markers 1 4.829e-06 4.829e-06 > org-agenda-deadline-face 4 4.761e-06 1.19025e-06 > org-agenda-update-agenda-type 1 1.575e-06 1.575e-06 > org-agenda-ndays-to-span 2 1.174e-06 5.87e-07 > org-agenda-mark-clocking-task 1 7.16e-07 7.16e-07 > org-agenda-use-sticky-p 1 5.02e-07 5.02e-07 | > > I’m not sure if that says much though. > > Version information: > > * > > Emacs: |GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ > Version 3.18.9) of 2018-02-18| > > * > > Orgmode: |Org mode version 9.1.6 (9.1.6-44-ge6f891-elpaplus @ > /home/jx/.emacs.d/elpa/develop/org-plus-contrib-20180205/)|. > > * > > I’m using Spacemacs’ > develop branch. > > However, this problem happens on both Ubuntu 16.04 LTS and the > latest MacOS. Actually it has persisted for a very long period of > time so I doubt if it’s related to any particular Emacs/Org version. > > Any help would be much appreciated! > > Best regards, > > Xiang > > ​ > > --------------D7698C964DD686204A49B04A Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

I finally found out the problem. I've been using Spacemacs and the `spell-checking` layer would automatically initiate flyspell on each org file, resulting in tremendous delay. If I disable flyspell from being automatically activated, the agenda is shown in a few seconds.


On 26.02.2018 17:29, Daniel Clemente wrote:
It can be many things. My emacs was taking 6 minutes to open for some years, due to many reasons: a slow computer, many org files and very large (>100 files, >25 Mb in total), lack of optimizations from my side and from org-mode's side. I got used to it but it's a very bad experience, specially when it crashes (and I made it crash a lot).
Now org-mode has improved and I disabled the slow parts, and it opens very fast (<20 seconds). I still open large files. 

The problem may not only be in org but in emacs (e.g. vc-mode can be slow), though from the instrumentation you send, it doesn't seem to be the case because it doesn't spend time opening the files.

org-agenda-prepare-buffers is the slow function, so it's the one you should check. Try to use edebug in it (C-u M-C-x) and run it slowly to see in which section it's the slowest. Then optimize that part. You'll see many optimization there (ignore properties, etc.). I had to disable many slow things, like <<<radios>>> (when you use a lot, e.g. 1000 per file, it's too slow and even crashes), I think that these radio targets were the main reason of the slowness.

Try creating a very large file and then check opening it, to see if it's also slow.
Try things in a clean emacs (e.g. starting with: emacs -Q), because you may have some other mode enabled that wants to do lots of things when you open the files.

You can test many more things. But keep testing, because it's possible to make it work faster.


On Mon, Feb 19, 2018 at 3:00 AM, JI Xiang <jimmyjx51@gmail.com> wrote:

Hello, first time posting on the mailing list. I encountered a problem where my agenda view (C-c a a) is taking ridiculously long (nearly 2 minutes) to be shown the first time. From the second time onwards, the time is about 11 seconds, which is still very long by any means.

I wondered whether it has something to do with me having many files in the agenda list, so I tried to restrict the list to just one file. But still the first call to org-agenda-list took 12 seconds. This shouldn’t be normal, right?

The following are shown in the Messages buffer during the call. 

Press key for agenda command: 
Restoring clock data 
Loading /home/jx/.emacs.d/.cache/org-clock-save.el (source)...done 
[yas] Prepared just-in-time loading of snippets successfully. 
Importmagic and/or epc not found. importmagic.el will not be working. 
Setting up indent for shell type zsh 
Indentation variables are now local. 
Indentation setup for shell type zsh 
Using vacuous schema 
Shell native completion is disabled, using fallback

I’m not sure why they would be there, especially the “using vacuous schema” and “shell native completion” part. Are the messages related to some #BEGIN_SRC blocks in the org files? I don’t think I would ever need source code blocks when I’m viewing an agenda buffer?

I posted the question on Emacs.SE and somebody suggested me use elp-instrument to perform a profiling. The results are as follows: 


    Function Name, Call Count, Elapsed Time, Average Time 

    org-agenda 1           116.6048159   116.6048159 
    org-agenda-list 1           116.29427357  116.29427357 
    org-agenda-prepare 1           109.15345470  109.15345470 
    org-agenda-prepare-buffers 1           108.98258905  108.98258905 
    org-agenda-get-day-entries 1288        7.0089191339  0.0054417074 
    org-agenda-get-scheduled 1288        3.726361062   0.0028931374 
    org-agenda-get-deadlines 1288        2.1579713230  0.0016754435 
    org-agenda--timestamp-to-absolute 14544       1.1317418120   7.781...e-05 
    org-agenda-get-timestamps 1288        0.3673404320  0.0002852021 
    org-agenda-get-sexps 1288        0.3438970410  0.0002670008 
    org-agenda-get-restriction-and-command 1           0.310503975    0.310503975 
    org-agenda-get-blocks 1288        0.3083237900  0.0002393818 
    org-agenda-prepare-window 1           0.157091024   0.157091024 
    org-agenda-skip 8624        0.0781296389  9.059...e-06 
    org-agenda-files 49          0.026090686   0.0005324629 
    org-agenda-finalize 1           0.013325178   0.013325178 
    org-agenda-mode 1           0.009848546   0.009848546 
    org-agenda-finalize-entries 23          0.006178961   0.0002686504 
    org-agenda-today-p 2604        0.0061717839  2.370...e-06 
    org-agenda-skip-eval 16968       0.0052723920  3.107...e-07 
    org-agenda-highlight-todo 112         0.0039840550  3.557...e-05 
    org-agenda-format-item 112         0.0038309749  3.420...e-05 
    org-agenda-new-marker 208         0.0031529359  1.515...e-05 
    org-agenda-format-date-aligned 28          0.0007373120  2.633...e-05 
    org-agenda-add-time-grid-maybe 28          0.0003810809  1.361...e-05 
    org-agenda-fix-displayed-tags 112         0.000342384   3.057e-06 
    org-agenda-fontify-priorities 1           0.00018465    0.00018465 
    org-agenda-align-tags 1           0.000160119   0.000160119 
    org-agenda-get-day-face 28          0.000156385   5.585...e-06 
    org-agenda-get-category-icon 112         7.702...e-05  6.877...e-07 
    org-agenda-span-name 23          4.4484e-05    1.934...e-06 
    org-agenda-span-to-ndays 1           2.3496e-05    2.3496e-05 
    org-agenda-fit-window-to-buffer 1           2.0877e-05    2.0877e-05 
    org-agenda-time-of-day-to-ampm-maybe 21          1.124...e-05   5.353...e-07 
    org-agenda-set-mode-name 1           9.226e-06     9.226e-06 
    org-agenda-mark-header-line 1           8.774e-06     8.774e-06 
    org-agenda-reset-markers 1           4.829e-06     4.829e-06 
    org-agenda-deadline-face 4           4.761e-06     1.19025e-06 
    org-agenda-update-agenda-type 1           1.575e-06     1.575e-06 
    org-agenda-ndays-to-span 2           1.174e-06     5.87e-07 
    org-agenda-mark-clocking-task 1           7.16e-07      7.16e-07 
    org-agenda-use-sticky-p 1           5.02e-07      5.02e-07

I’m not sure if that says much though.

Version information:

  • Emacs: GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.18.9) of 2018-02-18

  • Orgmode: Org mode version 9.1.6 (9.1.6-44-ge6f891-elpaplus @ /home/jx/.emacs.d/elpa/develop/org-plus-contrib-20180205/).

  • I’m using Spacemacs’ develop branch.

However, this problem happens on both Ubuntu 16.04 LTS and the latest MacOS. Actually it has persisted for a very long period of time so I doubt if it’s related to any particular Emacs/Org version.

Any help would be much appreciated!

Best regards,

Xiang

​


--------------D7698C964DD686204A49B04A--