From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Clemente Subject: Re: Ridiculously long (2 minutes) startup time for org agenda Date: Mon, 26 Feb 2018 18:29:23 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000005010810566200320" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:58176) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqLeZ-00072M-OC for emacs-orgmode@gnu.org; Mon, 26 Feb 2018 11:29:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqLeX-0008K9-7w for emacs-orgmode@gnu.org; Mon, 26 Feb 2018 11:29:27 -0500 Received: from mail-yb0-x231.google.com ([2607:f8b0:4002:c09::231]:37723) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eqLeW-0008Js-V5 for emacs-orgmode@gnu.org; Mon, 26 Feb 2018 11:29:25 -0500 Received: by mail-yb0-x231.google.com with SMTP id u5-v6so4208078ybf.4 for ; Mon, 26 Feb 2018 08:29:24 -0800 (PST) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: JI Xiang Cc: Org-mode Org-Mode --0000000000005010810566200320 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable It can be many things. My emacs was taking 6 minutes to open for some years, due to many reasons: a slow computer, many org files and very large (>100 files, >25 Mb in total), lack of optimizations from my side and from org-mode's side. I got used to it but it's a very bad experience, specially when it crashes (and I made it crash a lot). Now org-mode has improved and I disabled the slow parts, and it opens very fast (<20 seconds). I still open large files. The problem may not only be in org but in emacs (e.g. vc-mode can be slow), though from the instrumentation you send, it doesn't seem to be the case because it doesn't spend time opening the files. org-agenda-prepare-buffers is the slow function, so it's the one you should check. Try to use edebug in it (C-u M-C-x) and run it slowly to see in which section it's the slowest. Then optimize that part. You'll see many optimization there (ignore properties, etc.). I had to disable many slow things, like <<>> (when you use a lot, e.g. 1000 per file, it's too slow and even crashes), I think that these radio targets were the main reason of the slowness. Try creating a very large file and then check opening it, to see if it's also slow. Try things in a clean emacs (e.g. starting with: emacs -Q), because you may have some other mode enabled that wants to do lots of things when you open the files. You can test many more things. But keep testing, because it's possible to make it work faster. On Mon, Feb 19, 2018 at 3:00 AM, JI Xiang wrote: > Hello, first time posting on the mailing list. I encountered a problem > where my agenda view (C-c a a) is taking ridiculously long (nearly 2 > minutes) to be shown the first time. From the second time onwards, the ti= me > is about 11 seconds, which is still very long by any means. > > I wondered whether it has something to do with me having many files in th= e > agenda list, so I tried to restrict the list to just one file. But still > the first call to org-agenda-list took 12 seconds. This shouldn=E2=80=99t= be > normal, right? > > The following are shown in the Messages buffer during the call. > > Press key for agenda command: > Restoring clock data > Loading /home/jx/.emacs.d/.cache/org-clock-save.el (source)...done > [yas] Prepared just-in-time loading of snippets successfully. > Importmagic and/or epc not found. importmagic.el will not be working. > Setting up indent for shell type zsh > Indentation variables are now local. > Indentation setup for shell type zsh > Using vacuous schema > Shell native completion is disabled, using fallback > > I=E2=80=99m not sure why they would be there, especially the =E2=80=9Cusi= ng vacuous > schema=E2=80=9D and =E2=80=9Cshell native completion=E2=80=9D part. Are t= he messages related to > some #BEGIN_SRC blocks in the org files? I don=E2=80=99t think I would ev= er need > source code blocks when I=E2=80=99m viewing an agenda buffer? > > I posted the question on Emacs.SE > > and somebody suggested me use elp-instrument to perform a profiling. The > results are as follows: > > > Function Name, Call Count, Elapsed Time, Average Time > > org-agenda 1 116.6048159 116.6048159 > org-agenda-list 1 116.29427357 116.29427357 > org-agenda-prepare 1 109.15345470 109.15345470 > org-agenda-prepare-buffers 1 108.98258905 108.98258905 > org-agenda-get-day-entries 1288 7.0089191339 0.0054417074 > org-agenda-get-scheduled 1288 3.726361062 0.0028931374 > org-agenda-get-deadlines 1288 2.1579713230 0.0016754435 > org-agenda--timestamp-to-absolute 14544 1.1317418120 7.781...= e-05 > org-agenda-get-timestamps 1288 0.3673404320 0.0002852021 > org-agenda-get-sexps 1288 0.3438970410 0.0002670008 > org-agenda-get-restriction-and-command 1 0.310503975 0.3= 10503975 > org-agenda-get-blocks 1288 0.3083237900 0.0002393818 > org-agenda-prepare-window 1 0.157091024 0.157091024 > org-agenda-skip 8624 0.0781296389 9.059...e-06 > org-agenda-files 49 0.026090686 0.0005324629 > org-agenda-finalize 1 0.013325178 0.013325178 > org-agenda-mode 1 0.009848546 0.009848546 > org-agenda-finalize-entries 23 0.006178961 0.0002686504 > org-agenda-today-p 2604 0.0061717839 2.370...e-06 > org-agenda-skip-eval 16968 0.0052723920 3.107...e-07 > org-agenda-highlight-todo 112 0.0039840550 3.557...e-05 > org-agenda-format-item 112 0.0038309749 3.420...e-05 > org-agenda-new-marker 208 0.0031529359 1.515...e-05 > org-agenda-format-date-aligned 28 0.0007373120 2.633...e-05 > org-agenda-add-time-grid-maybe 28 0.0003810809 1.361...e-05 > org-agenda-fix-displayed-tags 112 0.000342384 3.057e-06 > org-agenda-fontify-priorities 1 0.00018465 0.00018465 > org-agenda-align-tags 1 0.000160119 0.000160119 > org-agenda-get-day-face 28 0.000156385 5.585...e-06 > org-agenda-get-category-icon 112 7.702...e-05 6.877...e-07 > org-agenda-span-name 23 4.4484e-05 1.934...e-06 > org-agenda-span-to-ndays 1 2.3496e-05 2.3496e-05 > org-agenda-fit-window-to-buffer 1 2.0877e-05 2.0877e-05 > org-agenda-time-of-day-to-ampm-maybe 21 1.124...e-05 5.353= ...e-07 > org-agenda-set-mode-name 1 9.226e-06 9.226e-06 > org-agenda-mark-header-line 1 8.774e-06 8.774e-06 > org-agenda-reset-markers 1 4.829e-06 4.829e-06 > org-agenda-deadline-face 4 4.761e-06 1.19025e-06 > org-agenda-update-agenda-type 1 1.575e-06 1.575e-06 > org-agenda-ndays-to-span 2 1.174e-06 5.87e-07 > org-agenda-mark-clocking-task 1 7.16e-07 7.16e-07 > org-agenda-use-sticky-p 1 5.02e-07 5.02e-07 > > I=E2=80=99m not sure if that says much though. > > Version information: > > - > > Emacs: GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version > 3.18.9) of 2018-02-18 > - > > Orgmode: Org mode version 9.1.6 (9.1.6-44-ge6f891-elpaplus @ > /home/jx/.emacs.d/elpa/develop/org-plus-contrib-20180205/). > - > > I=E2=80=99m using Spacemacs=E2=80=99 develop > branch. > > However, this problem happens on both Ubuntu 16.04 LTS and the latest > MacOS. Actually it has persisted for a very long period of time so I doub= t > if it=E2=80=99s related to any particular Emacs/Org version. > > Any help would be much appreciated! > > Best regards, > > Xiang > =E2=80=8B > --0000000000005010810566200320 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
It can be many things. My emacs was taking 6 minutes to op= en for some years, due to many reasons: a slow computer, many org files and= very large (>100 files, >25 Mb in total), lack of optimizations from= my side and from org-mode's side. I got used to it but it's a very= bad experience, specially when it crashes (and I made it crash a lot).Now org-mode has improved and I disabled the slow parts, and it opens very= fast (<20 seconds). I still open large files.=C2=A0

The pr= oblem may not only be in org but in emacs (e.g. vc-mode can be slow), thoug= h from the instrumentation you send, it doesn't seem to be the case bec= ause it doesn't spend time opening the files.

= org-agenda-prepare-buffers is the slow function, so it's the one you sh= ould check. Try to use edebug in it (C-u M-C-x) and run it slowly to see in= which section it's the slowest. Then optimize that part. You'll se= e many optimization there (ignore properties, etc.). I had to disable many = slow things, like <<<radios>>> (when you use a lot, e.g. = 1000 per file, it's too slow and even crashes), I think that these radi= o targets were the main reason of the slowness.

Tr= y creating a very large file and then check opening it, to see if it's = also slow.
Try things in a clean emacs (e.g. starting with: emacs= -Q), because you may have some other mode enabled that wants to do lots of= things when you open the files.

You can= test many more things. But keep testing, because it's possible to make= it work faster.


On Mon, Feb 19, 2018 at 3:00 AM, JI Xiang <= span dir=3D"ltr"><jimmyjx51@gmail.com> wrote:
=20 =20 =20

Hello, first time posting on the mailing list. I encountered a problem where my agenda view (C-c a a) is taking ridiculously long (nearly 2 minutes) to be shown the first time. From the second time onwards, the time is about 11 seconds, which is still very long by any means.

I wondered whether it has something to do with me having many files in the agenda list, so I tried to restrict the list to just one file. But still the first call to org-agenda-list took 12 seconds. This shouldn=E2=80=99t be normal, right?

The following are shown in the Messages buffer during the call. 

Press key for=
 agenda command:=20
Restoring clock data=20
Loading /home/jx/.emacs.d/.cache/org-clock-save.el (source)...done=20
[yas] Prepared just-in-time loading of snippets successfully.=20
Importmagic and/or epc not found. importmagic.el will not be working.=20
Setting up indent for shell type zsh=20
Indentation variables are now local.=20
Indentation setup for shell type zsh=20
Using vacuous schema=20
Shell native completion is disabled, using fallback

I=E2=80=99m not sure why they would be there, especially the =E2=80=9Cusing vacuous schema= =E2=80=9D and =E2=80=9Cshell native completion=E2=80=9D part. Are the messages re= lated to some #BEGIN_SRC blocks in the org files? I don=E2=80=99t think I would ever need so= urce code blocks when I=E2=80=99m viewing an agenda buffer?

I posted the question on Emacs.SE and somebody suggested me use elp-i= nstrument to perform a profiling. The results are as follows: 


    Function Name, Call Count, Elapsed Time, Average Time=20

    org-agenda 1           116.6048159   116.6048159=20
    org-agenda-list 1           116.29427357  116.29427357=20
    org-agenda-prepare 1           109.15345470  109.15345470=20
    org-agenda-prepare-buffers 1           108.98258905  108.98258905=20
    org-agenda-get-day-entries 1288        7.0089191339  0.0054417074=20
    org-agenda-get-scheduled 1288        3.726361062   0.0028931374=20
    org-agenda-get-deadlines 1288        2.1579713230  0.0016754435=20
    org-agenda--timestamp-to-absolute 14544       1.1317418120   7.781=
...e-05=20
    org-agenda-get-timestamps 1288        0.3673404320  0.0002852021=20
    org-agenda-get-sexps 1288        0.3438970410  0.0002670008=20
    org-agenda-get-restriction-and-command 1           0.310503975    =
0.310503975=20
    org-agenda-get-blocks 1288        0.3083237900  0.0002393818=20
    org-agenda-prepare-window 1           0.157091024   0.157091024=20
    org-agenda-skip 8624        0.0781296389  9.059...e-06=20
    org-agenda-files 49          0.026090686   0.0005324629=20
    org-agenda-finalize 1           0.013325178   0.013325178=20
    org-agenda-mode 1           0.009848546   0.009848546=20
    org-agenda-finalize-entries 23          0.006178961   0.0002686504=20
    org-agenda-today-p 2604        0.0061717839  2.370...e-06=20
    org-agenda-skip-eval 16968       0.0052723920  3.107...e-07=20
    org-agenda-highlight-todo 112         0.0039840550  3.557...e-05=20
    org-agenda-format-item 112         0.0038309749  3.420...e-05=20
    org-agenda-new-marker 208         0.0031529359  1.515...e-05=20
    org-agenda-format-date-aligned 28          0.0007373120  2.633...e-05=
=20
    org-agenda-add-time-grid-maybe 28          0.0003810809  1.361...e-05=
=20
    org-agenda-fix-displayed-tags 112         0.000342384   3.057e-06=20
    org-agenda-fontify-priorities 1           0.00018465    0.00018465=20
    org-agenda-align-tags 1           0.000160119   0.000160119=20
    org-agenda-get-day-face 28          0.000156385   5.585...e-06=20
    org-agenda-get-category-icon 112         7.702...e-05  6.877...e-07=20
    org-agenda-span-name 23          4.4484e-05    1.934...e-06=20
    org-agenda-span-to-ndays 1           2.3496e-05    2.3496e-05=20
    org-agenda-fit-window-to-buffer 1           2.0877e-05    2.0877e-=
05=20
    org-agenda-time-of-day-to-ampm-maybe 21          1.124...e-05   5.=
353...e-07=20
    org-agenda-set-mode-name 1           9.226e-06     9.226e-06=20
    org-agenda-mark-header-line 1           8.774e-06     8.774e-06=20
    org-agenda-reset-markers 1           4.829e-06     4.829e-06=20
    org-agenda-deadline-face 4           4.761e-06     1.19025e-06=20
    org-agenda-update-agenda-type 1           1.575e-06     1.575e-06=20
    org-agenda-ndays-to-span 2           1.174e-06     5.87e-07=20
    org-agenda-mark-clocking-task 1           7.16e-07      7.16e-07=20
    org-agenda-use-sticky-p 1           5.02e-07      5.02e-07

I=E2=80=99m not sure if t= hat says much though.

Version information:

  • Emacs: GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-g= nu, GTK+ Version 3.18.9) of 2018-02-18

  • Orgmode: Org mode version 9.1.6 (9.1.6-44-ge6f891-el= paplus @ /home/jx/.emacs.d/elpa/develop/org-plus-contrib-2018020= 5/).

  • I=E2=80=99m using Spacemacs=E2=80=99 develop branch.

However, this problem happens on both Ubuntu 16.04 LTS and the latest MacOS. Actually it has persisted for a very long period of time so I doubt if it=E2=80=99s related to any particular Emacs/Org version. =

Any help would be much appreciated!

Best regards,

Xiang

=E2=80=8B

--0000000000005010810566200320--