Asynchronous org-babel-tangle (was Re: Asynchronous org-agenda-redo)

[Diego Zamboni <diego@zzamboni.org>]

[-- Attachment #1: Type: text/plain, Size: 7593 bytes --]

Hi Ihor,

On Thu, Dec 12, 2019 at 4:00 PM Ihor Radchenko <yantar92@gmail.com> wrote:

> See the relevant code from my config below. Let me know if you have any
> questions or suggestions.
>

Thank you, this was extremely helpful. I am now using a modified version of
your code (see below) with the following comments/questions:

- For me, the tangle works if I only load the org library in the
sub-process, so I disabled the loading of my config. I guess this depends
heavily and what other configuration one has loaded, but I think for
tangling, not much else is needed :)

- In you code, the process-list variable was not actually being modified
since plist-put returns the new value but does not modify the variable. I
tried a few variations wrapping the plist-put in (setq ...), and adding
corresponding code to clear the element after the tangle was finished, but
could not get it to work. As this is not really critical for me, I just
removed the whole process-checking code, significantly simplifying the rest.

- I wanted my code to report the time the tangle took. This works well by
passing the time as the result of the initial lambda. Interestingly, it
works if I return the time already formatted as a string (as in my code)
but not if I return the result of (float-time ..) directly.

Here's my code:

  #+begin_src emacs-lisp
    (defun zz/org-babel-tangle-async (file)
      "Invoke `org-babel-tangle-file' asynchronously."
      (message "Tangling %s..." (buffer-file-name))
      (async-start
       (let ((args (list file)))
         `(lambda ()
            (require 'org)
            ;;(load "~/.emacs.d/init.el")
            (let ((start-time (current-time)))
              (apply #'org-babel-tangle-file ',args)
              (format "%.2f" (float-time (time-since start-time))))))
       (let ((message-string (format "Tangling %S completed after " file)))
         `(lambda (tangle-time)
            (message (concat ,message-string
                             (format "%s seconds" tangle-time)))))))

    (defun zz/org-babel-tangle-current-buffer-async ()
      "Tangle current buffer asynchronously."
      (zz/org-babel-tangle-async (buffer-file-name)))
    #+end_src

Thanks again for your help, and for the inspiration to finally get this
done, it had been bugging me for a while :)

--Diego



> Also, my config is written in the way that everything related to user
> interaction can be disabled if I load emacs for tangling
> (with org-tangle-flag). This is to speed up the async process.
>
> #+begin_src emacs-lisp
> (defvar yant/org-babel-tangle-async-process-list nil
>   "Plist of (file . process) for all the currently running async tangle
> processes.")
>
>
> (defun yant/org-babel-tangle-async (file &optional target-file lang)
>   "Invoke `org-babel-tangle-file' asynchronously."
>   (require 'async)
>   (let ((oldproc (plist-get yant/org-babel-tangle-async-process-list
> file)))
>     (when (or (not oldproc)
>               (async-wait oldproc))
>       (message "Tangling %s..." (buffer-file-name))
>       (plist-put yant/org-babel-tangle-async-process-list
>                  file
>                  (async-start
>                   (let ((args (list file target-file lang)))
>                     `(lambda ()
>                        (require 'org)
>                        (setq org-tangle-flag t)
>                        (load "~/.emacs.d/config.el")
>                        (apply #'org-babel-tangle-file ',args)))
>                   (let ((message-string (format "Tangling (%S %S %S)
> completed." file target-file lang)))
>                     `(lambda (result) (message ,message-string))))))))
>
> (defvar yant/auto-tangle-list nil
>   "List of files, which can be safely tangled on save.
> The list is saved between Emacs sessions.")
>
> (when init-flag
>   (use-package savehist
>     :config
>     (add-to-list 'savehist-additional-variables 'yant/auto-tangle-list))
>   (savehist-mode +1)
>   (defun yant/toggle-buffer-auto-tangle (arg)
>     "Toggle auto tangling of a buffer."
>     (interactive "P")
>     (if (not (eq major-mode 'org-mode))
>         (message "Org-mode is not active in buffer \"%s\"" (buffer-name))
>       (cond ((not arg)
>              (if (member (buffer-file-name) yant/auto-tangle-list)
>                  (progn (setq yant/auto-tangle-list (delete
> (buffer-file-name) yant/auto-tangle-list))
>                         (message "Auto tangling disabled for %s"
> (buffer-file-name)))
>                (add-to-list 'yant/auto-tangle-list (buffer-file-name))
>                (message "Auto tangling enabled for %s"
> (buffer-file-name))))
>             ((or (and (not (listp arg)) (> arg 0))
>                  (equal arg '(4)))
>              (add-to-list 'yant/auto-tangle-list (buffer-file-name))
>              (message "Auto tangling enabled for %s" (buffer-file-name)))
>             (t
>              (setq yant/auto-tangle-list (delete (buffer-file-name)
> yant/auto-tangle-list))
>              (message "Auto tangling disabled for %s"
> (buffer-file-name))))))
>
>   (bind-key "C-c C-*" #'yant/toggle-buffer-auto-tangle org-mode-map))
>
> (defun yant/org-babel-tangle-current-buffer-async ()
>   "Tangle current buffer asynchronously."
>   (when (and (eq major-mode 'org-mode)
>              (member (buffer-file-name) yant/auto-tangle-list))
>     (yant/org-babel-tangle-async (buffer-file-name))))
>
> (add-hook 'after-save-hook #'yant/org-babel-tangle-current-buffer-async)
> #+end_src
>
>
> Diego Zamboni <diego@zzamboni.org> writes:
>
> > Hi Ihor,
> >
> > I cannot answer your question, but I am curious about using async
> together
> > with tangling, since for some of my buffers, tangling takes some time and
> > freezes Emacs in the process. Do you have some examples of this that you
> > could share?
> >
> > Thanks,
> > --Diego
> >
> >
> > On Thu, Dec 12, 2019 at 9:21 AM Ihor Radchenko <yantar92@gmail.com>
> wrote:
> >
> >> I am thinking if it is possible to implement org-agenda-redo
> >> asynchronously.
> >>
> >> Rebuilding agenda should normally not affect any buffer except agenda
> >> buffer. So, it should be sufficient to block any agenda modifying
> >> commands in the agenda buffer, redo the agenda buffer in separate
> >> thread, and replace the old agenda with the calculated one.
> >> Then, emacs should remain responsive while updating agenda (except for
> >> modifying the agenda buffer).
> >>
> >> For example, this naive code kind of works (forgetting that buffer-local
> >> variables will not be passed to the thread):
> >>
> >> (define-advice org-agenda-redo (:around (oldfun &optional all)
> make-async)
> >>   (make-thread oldfun "org-agenda-redo"))
> >>
> >> The problem is that emacs does not become responsive...
> >>
> >> Another approach would be using async.el package, which allows calling
> >> arbitrary function in subordinate emacs process. Then, the main emacs
> >> instance should not be "frozen" (I use same approach for tangling and it
> >> works fine).
> >>
> >> However, the question is how to pass the .org and agenda buffers to this
> >> subordinate process. Opening the .org files there is not a good option
> >> since it would give too much overhead to this asynchronous agenda.
> >>
> >> Any suggestions? Alternative ideas?
> >>
> >> Best,
> >> Ihor
> >>
> >>
> >>
>
> --
> Ihor Radchenko,
> PhD,
> Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
> State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong
> University, Xi'an, China
> Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg
>

[-- Attachment #2: Type: text/html, Size: 9846 bytes --]