Hack: org-agenda-cache.el

Hack: org-agenda-cache.el

[Max Mikhanosha]

[-- Attachment #1: Type: text/plain, Size: 1426 bytes --]

As my agenda files slowly grew over the years, the speed of generating
agenda had slowly deteriorated, finally hitting the point of me jdoing
something about it. (it got to around 5-7 seconds, which kind of
interrupts work-flow)

Attached file org-agenda-cache.el is a "quick fix" solution that I
developed for myself, its more of a request for discussion and not
intended as contribution yet.

Basic idea is:

1. Multiple-agenda buffers can exist at the same time, having separate
tag filters and other such settings. This is accomplished by bunch of
org-agenda-* variables being made buffer-local.

2. Custom agenda commands should bind `org-agenda-buffer-name' variable,
so that for example C-c a generates "*Agenda*" buffer and C-c t
generates "*Todo List*" buffer. 

3. org-agenda checks if buffer with `org-agenda-buffer-name' exists
and if it does, it will show that buffer, rather then re-generating
it. To generate a fresh agenda, becomes C-c a r instead of C-c a

4. org-agenda-quit buries the agenda buffer instead of killing it.

5. All of the above shenanigans can be switched on and off by doing
M-x toggle-org-agenda-caching command, its off by default so you have
to turn it on after loading the file.

I had been using this setup for last few days, and it had really been
a blast, returning me to the times when I just started with org-mode,
and information had appeared in milliseconds rather then seconds.


[-- Attachment #2: org-agenda-cache.el --]
[-- Type: application/octet-stream, Size: 5668 bytes --]

;;; org-agenda-cache.el --- Cache agenda buffers instead of regenerating them each time

;; Cached agenda buffers
;;
;; Toggle with M-x toggle-org-agenda-caching
;;
;; If you include `org-agenda-buffer-name' into the list of bound
;; variables for a specific agenda view, then these agenda views
;; will be cached independently each in their own buffer
;;
;; Example setup:
;;
;; (setq org-agenda-custom-commands
;;       '(("a" "Agenda and NEXT (priority)"
;;          ((agenda ""
;;                   ((org-agenda-span 'day)))
;;           (tags-todo "/!NEXT"
;;                      ((org-agenda-overriding-header "Next Tasks")
;;                       (org-agenda-tags-todo-honor-ignore-options t)
;;                       (org-agenda-todo-ignore-scheduled t)
;;                       (org-agenda-todo-ignore-deadlines t)
;;                       (org-tags-match-list-sublevels t)
;;                       (org-agenda-sorting-strategy
;;                        '(priority-down category-keep)))))
;;          ((org-agenda-buffer-name "*Agenda*")))
;;         ("t" "TODO entries"
;;          todo ""
;;          ((org-agenda-buffer-name "*Todo List*")))))
;;
;; Note that C-c a and C-c t will use two different agenda buffers
;; that are indecent of each other, and each can have different restrictions
;; or tag filter

(defvar org-agenda-use-caching nil
  "When non-nil, org-agenda will show existing agenda buffer when
its available and org-agenda-exit will bury the agenda buffer
instead of destroying it. ")

(defun toggle-org-agenda-caching (&optional arg)
  (interactive)
  (setq org-agenda-use-caching (or arg (not org-agenda-use-caching)))
  (message "Agenda caching was %s" (if org-agenda-use-caching "enabled" "disabled")))

(defvar org-agenda-inside-org-agenda nil)
(defvar org-cached-agenda-last-prefix-arg nil)
(defvar org-this-agenda-buffer-name nil)

(dolist (var '(org-cached-agenda-last-prefix-arg
               org-this-agenda-buffer-name
               org-agenda-redo-command
               org-agenda-query-string
               org-agenda-tag-filter
               org-agenda-tag-filter-overlays
               org-agenda-category-filter
               org-agenda-columns-active
               org-agenda-marker-table
               org-agenda-markers
               org-agenda-restrict
               org-agenda-restrict-begin
               org-agenda-overriding-restriction
               org-agenda-archives-mode
               org-agenda-current-span))
  (make-variable-buffer-local var))

(defadvice org-agenda (around cached-agenda-buffer activate)
  (if org-agenda-use-caching
      (let ((org-agenda-inside-org-agenda t))
        (catch 'org-agenda-used-existing-buffer
          (setq ad-return-value ad-do-it)))
    (setq ad-return-value ad-do-it)))

(defadvice org-agenda-redo (around cached-agenda-buffer activate)
  (if org-agenda-use-caching
      (let ((org-agenda-buffer-name (or org-this-agenda-buffer-name
                                        org-agenda-buffer-name)))
        (setq ad-return-value ad-do-it))
    (setq ad-return-value ad-do-it)))

(defadvice org-prepare-agenda (around cached-agenda-buffer activate)
  (if org-agenda-use-caching
      (if (or (not org-agenda-inside-org-agenda)
              org-agenda-multi
              (not (get-buffer org-agenda-buffer-name))
              (with-current-buffer (get-buffer org-agenda-buffer-name)
                (not (equal
                      current-prefix-arg
                      org-cached-agenda-last-prefix-arg))))
          (progn
            (setq ad-return-value ad-do-it)
            (with-current-buffer (get-buffer org-agenda-buffer-name)
              (setq org-cached-agenda-last-prefix-arg current-prefix-arg)
              (setq org-this-agenda-buffer-name org-agenda-buffer-name)))
        ;; Below block cut-n-paste copied from real `org-prepare-agenda' function
        (let* ((abuf (get-buffer-create org-agenda-buffer-name))
               (awin (get-buffer-window abuf)))
          (cond
           ((equal (current-buffer) abuf) nil)
           (awin (select-window awin))
           ((not (setq org-pre-agenda-window-conf (current-window-configuration))))
           ((equal org-agenda-window-setup 'current-window)
            (org-pop-to-buffer-same-window abuf))
           ((equal org-agenda-window-setup 'other-window)
            (org-switch-to-buffer-other-window abuf))
           ((equal org-agenda-window-setup 'other-frame)
            (switch-to-buffer-other-frame abuf))
           ((equal org-agenda-window-setup 'reorganize-frame)
            (delete-other-windows)
            (org-switch-to-buffer-other-window abuf)))
          ;; additional test in case agenda is invoked from within agenda
          ;; buffer via elisp link
          (unless (equal (current-buffer) abuf)
            (org-pop-to-buffer-same-window abuf))
          ;; make org-agenda exit early
          (throw 'org-agenda-used-existing-buffer nil)))
    (setq ad-return-value ad-do-it)))

(defadvice org-agenda-quit (around cached-agenda-buffer activate)
  (if org-agenda-use-caching
      (org-agenda-bury)
    (setq ad-return-value ad-do-it)))

(defun org-agenda-bury ()
  "Restore window or frame configuration just like `org-agenda-quit' does
but bury the agenda buffer rather then killing it."
  (interactive)
  (let ((buf (current-buffer)))
    (if (eq org-agenda-window-setup 'other-frame)
        (progn
          (delete-frame))
      (and (not (eq org-agenda-window-setup 'current-window))
           (not (one-window-p))
           (delete-window)))
    (with-current-buffer buf
        (bury-buffer))))

(provide 'org-agenda-cache)

[-- Attachment #3: Type: text/plain, Size: 1 bytes --]

Re: Hack: org-agenda-cache.el

[Carsten Dominik]

Hi Max,

this is a pretty good solution if the main issue with slow agenda
is that it feels slow because you are often switching back and forth
between different agenda buffers.  I like it, and I would vote for
integration into Org-mode, not as a separate package, but directly,
as an option.  If Bastien agrees, maybe you could make a patch which
introduces this into the main code.

Three comments:

1. I think there needs to be a command to remove all agenda buffers, to make sure that everything is updated.  Maybe this could be "C-u r" or so, which would rebuilt the current and remove all others.  This commands needs to make sure that all markers are safely discarded - otherwise you will die the marker death eventually.

2. I don't think that "-cache" is the best name choice for this, because this
is not a caching mechanism.  It is a "multiple-agenda-buffers" thing.

3. One caveat for this is that maybe in some places, org-agenda-marker-table
is modified while the agenda buffer is not current (I am not sure, I just think this needs to be checked).  If so, one would need to pop to the relevant agenda buffer to make the change.

Just my 2c, of course.

- Carsten
On 15.1.2012, at 20:47, Max Mikhanosha wrote:

> As my agenda files slowly grew over the years, the speed of generating
> agenda had slowly deteriorated, finally hitting the point of me jdoing
> something about it. (it got to around 5-7 seconds, which kind of
> interrupts work-flow)
> 
> Attached file org-agenda-cache.el is a "quick fix" solution that I
> developed for myself, its more of a request for discussion and not
> intended as contribution yet.
> 
> Basic idea is:
> 
> 1. Multiple-agenda buffers can exist at the same time, having separate
> tag filters and other such settings. This is accomplished by bunch of
> org-agenda-* variables being made buffer-local.
> 
> 2. Custom agenda commands should bind `org-agenda-buffer-name' variable,
> so that for example C-c a generates "*Agenda*" buffer and C-c t
> generates "*Todo List*" buffer. 
> 
> 3. org-agenda checks if buffer with `org-agenda-buffer-name' exists
> and if it does, it will show that buffer, rather then re-generating
> it. To generate a fresh agenda, becomes C-c a r instead of C-c a
> 
> 4. org-agenda-quit buries the agenda buffer instead of killing it.
> 
> 5. All of the above shenanigans can be switched on and off by doing
> M-x toggle-org-agenda-caching command, its off by default so you have
> to turn it on after loading the file.
> 
> I had been using this setup for last few days, and it had really been
> a blast, returning me to the times when I just started with org-mode,
> and information had appeared in milliseconds rather then seconds.
> 
> <org-agenda-cache.el>

Re: Hack: org-agenda-cache.el

[Carsten Dominik]

On 16.1.2012, at 13:18, Carsten Dominik wrote:

> Hi Max,
> 
> this is a pretty good solution if the main issue with slow agenda
> is that it feels slow because you are often switching back and forth
> between different agenda buffers.  I like it, and I would vote for
> integration into Org-mode, not as a separate package, but directly,
> as an option.  If Bastien agrees, maybe you could make a patch which
> introduces this into the main code.
> 
> Three comments:
> 
> 1. I think there needs to be a command to remove all agenda buffers, to make sure that everything is updated.  Maybe this could be "C-u r" or so, which would rebuilt the current and remove all others.  This commands needs to make sure that all markers are safely discarded - otherwise you will die the marker death eventually.
> 
> 2. I don't think that "-cache" is the best name choice for this, because this
> is not a caching mechanism.  It is a "multiple-agenda-buffers" thing.
> 
> 3. One caveat for this is that maybe in some places, org-agenda-marker-table
> is modified while the agenda buffer is not current (I am not sure, I just think this needs to be checked).  If so, one would need to pop to the relevant agenda buffer to make the change.

Sorry, I am talking about `org-agenda-markers', and indeed, some care has to be taken here.  Basically, the function org-agenda-new-marker needs to be modified to take
a buffer as an argument, and then to push the new marker on the list of markers in that buffer.  Also, org-agenda-save-markers-in-region needs to be modified.

OK, this is a bit complicated, I am willing to help.  If you make a new patch that implements the multibuffer stuff in org-agenda, I will look at the marker issues and implement them.

- Carsten

> 
> Just my 2c, of course.
> 
> - Carsten
> On 15.1.2012, at 20:47, Max Mikhanosha wrote:
> 
>> As my agenda files slowly grew over the years, the speed of generating
>> agenda had slowly deteriorated, finally hitting the point of me jdoing
>> something about it. (it got to around 5-7 seconds, which kind of
>> interrupts work-flow)
>> 
>> Attached file org-agenda-cache.el is a "quick fix" solution that I
>> developed for myself, its more of a request for discussion and not
>> intended as contribution yet.
>> 
>> Basic idea is:
>> 
>> 1. Multiple-agenda buffers can exist at the same time, having separate
>> tag filters and other such settings. This is accomplished by bunch of
>> org-agenda-* variables being made buffer-local.
>> 
>> 2. Custom agenda commands should bind `org-agenda-buffer-name' variable,
>> so that for example C-c a generates "*Agenda*" buffer and C-c t
>> generates "*Todo List*" buffer. 
>> 
>> 3. org-agenda checks if buffer with `org-agenda-buffer-name' exists
>> and if it does, it will show that buffer, rather then re-generating
>> it. To generate a fresh agenda, becomes C-c a r instead of C-c a
>> 
>> 4. org-agenda-quit buries the agenda buffer instead of killing it.
>> 
>> 5. All of the above shenanigans can be switched on and off by doing
>> M-x toggle-org-agenda-caching command, its off by default so you have
>> to turn it on after loading the file.
>> 
>> I had been using this setup for last few days, and it had really been
>> a blast, returning me to the times when I just started with org-mode,
>> and information had appeared in milliseconds rather then seconds.
>> 
>> <org-agenda-cache.el>
> 

Re: Hack: org-agenda-cache.el

[Max Mikhanosha]

Hi Carsten,

At Mon, 16 Jan 2012 16:40:01 +0100,
Carsten Dominik wrote:
> On 16.1.2012, at 13:18, Carsten Dominik wrote:
> 
> Sorry, I am talking about `org-agenda-markers', and indeed, some
> care has to be taken here.  Basically, the function
> org-agenda-new-marker needs to be modified to take a buffer as an
> argument, and then to push the new marker on the list of markers in
> that buffer.  Also, org-agenda-save-markers-in-region needs to be
> modified.
> 
> OK, this is a bit complicated, I am willing to help.  If you make a
> new patch that implements the multibuffer stuff in org-agenda, I
> will look at the marker issues and implement them.

Cool, I'll redo it and submit a patch in the next few days. The plan
is:

  - Multi-buffer support (ie buffer-local vars) goes in
    unconditionally, since I think there is no way to undo making
    variable buffer local, and if there is, it would make toggling on/off
    function really complicated.

    About the markers, what about if I call to clean them up
    kill-buffer-hook? This will work even if user kills the buffer
    manually, not just on q key.

  - The "use existing buffer, and bury on quit" toggle needs a new
    name org-agenda-reuse-buffers? Wanderlust has a similar mechanism
    and its called "sticky summary buffers", so maybe
    org-agenda-buffer-sticky?

  - The C-u r that you suggested to kill all buffers is already taken
    by "reload with a
    search string", so I think for cleanup bindings could be:

    q          -> kill buffer or bury if enabled
    Q          -> always kill
    C-u Q      -> kill all agenda buffers (ie cleanup)

    Now that I'm thinking it also needs to keep track of all agenda
    buffers, so need a global list. The kill-buffer hook should work
    there too.

Regards,
  Max

Re: Hack: org-agenda-cache.el

[Carsten Dominik]

On 16.1.2012, at 17:31, Max Mikhanosha wrote:

> Hi Carsten,
> 
> At Mon, 16 Jan 2012 16:40:01 +0100,
> Carsten Dominik wrote:
>> On 16.1.2012, at 13:18, Carsten Dominik wrote:
>> 
>> Sorry, I am talking about `org-agenda-markers', and indeed, some
>> care has to be taken here.  Basically, the function
>> org-agenda-new-marker needs to be modified to take a buffer as an
>> argument, and then to push the new marker on the list of markers in
>> that buffer.  Also, org-agenda-save-markers-in-region needs to be
>> modified.
>> 
>> OK, this is a bit complicated, I am willing to help.  If you make a
>> new patch that implements the multibuffer stuff in org-agenda, I
>> will look at the marker issues and implement them.
> 
> Cool, I'll redo it and submit a patch in the next few days. The plan
> is:
> 
>  - Multi-buffer support (ie buffer-local vars) goes in
>    unconditionally, since I think there is no way to undo making
>    variable buffer local, and if there is, it would make toggling on/off
>    function really complicated.

Not really, there is `kill-local-variable'.  But I agree, not big
reason to make it possible to turn it off.

> 
>    About the markers, what about if I call to clean them up
>    kill-buffer-hook? This will work even if user kills the buffer
>    manually, not just on q key.

Sounds good, make sure to add to the kill-buffer-hook *locally*,
using org-add-hook (for XEmacs compatibility).

> 
>  - The "use existing buffer, and bury on quit" toggle needs a new
>    name org-agenda-reuse-buffers? Wanderlust has a similar mechanism
>    and its called "sticky summary buffers", so maybe
>    org-agenda-buffer-sticky?

Sounds good to me.

> 
>  - The C-u r that you suggested to kill all buffers is already taken
>    by "reload with a
>    search string", so I think for cleanup bindings could be:
> 
>    q          -> kill buffer or bury if enabled
>    Q          -> always kill
>    C-u Q      -> kill all agenda buffers (ie cleanup)


Jup.

> 
>    Now that I'm thinking it also needs to keep track of all agenda
>    buffers, so need a global list. The kill-buffer hook should work
>    there too.

You can also loop over all buffers and find the ones which
are in org-agenda-mode.  This is a bit easier than keeping
track of them.

Regards

- Carsten