From: Matt Lundin <mdl@imapmail.org>
To: Org Mode <emacs-orgmode@gnu.org>
Subject: [PATCH] Fix and optimize publishing cache check
Date: Tue, 12 Aug 2014 08:09:16 -0500 [thread overview]
Message-ID: <87oavpyhvn.fsf@fastmail.fm> (raw)
[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]
This patch does two things:
1. It prevents org-publish from perpetually republishing files
containing includes. Currently, if an included file changes, it is
published, as is the file that includes it. However, until changes are
made in the file that includes it (which may be never) its modification
time remain older than that of the included file, so it will be
republished every time org-publish is called, even if neither file has
changed.
- Note: This patch fixes this behavior by updating the modification
time of the file that contains the includes. If this is deemed too
destructive/meddlesome, we could consider alternate behaviors, such
as setting the cached time with a simple float-time. Let me know
what would be best.
2. It optimizes checking for included files. Currently,
org-publish-cache-file-needs-publishing visits every file in a project
to check for includes. On my underpowered box, this takes a long time
(over a minute) on a project with 1000+ files, thus defeating the
purpose of the cache. This patch causes org-publish to store information
about included files in the cache itself:
a. If a file has changed it updates information about the files it
includes and stores it in the cache.
b. If a file has not changed, it checks the cache for included files
and sees if any of those files have been updated (thus preventing the
need to check every file in the project for includes)
Best,
Matt
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-and-optimize-publish-cache-check.patch --]
[-- Type: text/x-diff, Size: 4097 bytes --]
From 94a8061bb30e1992213fb8e71ee949d336d37435 Mon Sep 17 00:00:00 2001
From: Matt Lundin <mdl@imapmail.org>
Date: Tue, 12 Aug 2014 07:51:44 -0500
Subject: [PATCH] Fix and optimize publish cache check
* lisp/ox-publish.el: (org-publish-cache-file-needs-publishing) Fix
org-publish-cache-file-needs-publishing to change timestamp of files
containing includes. Speed up check for includes by storing
information about included files in cache itself.
This patch ensures that org-publish-cache-file-needs-publishing does
not keep publishing a file containing includes by updating the
modification time of that file. It also speeds up publishing by
caching information about included files, thus keeping
org-publish-cache-file-needs-publishing from having to visit every
file just to check includes (which can take a long time on a project
containing hundreds or thousands of files).
---
lisp/ox-publish.el | 60 +++++++++++++++++++++++++++++++-----------------------
1 file changed, 35 insertions(+), 25 deletions(-)
diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index df40572..228411f 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -1168,32 +1168,42 @@ the file including them will be republished as well."
(pstamp (org-publish-cache-get key))
(org-inhibit-startup t)
(visiting (find-buffer-visiting filename))
- included-files-ctime buf)
- (when (equal (file-name-extension filename) "org")
- (setq buf (find-file (expand-file-name filename)))
+ (ctime (org-publish-cache-ctime-of-src filename))
+ (needsp (or (null pstamp) (< pstamp ctime)))
+ includes buf)
+ ;; if the file needs publishing, refresh the included-files cache property
+ (when (and needsp
+ (equal (file-name-extension filename) "org"))
+ (setq buf (find-file-noselect (expand-file-name filename)))
(with-current-buffer buf
- (goto-char (point-min))
- (while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t)
- (let* ((element (org-element-at-point))
- (included-file
- (and (eq (org-element-type element) 'keyword)
- (let ((value (org-element-property :value element)))
- (and value
- (string-match "^\\(\".+?\"\\|\\S-+\\)" value)
- (org-remove-double-quotes
- (match-string 1 value)))))))
- (when included-file
- (add-to-list 'included-files-ctime
- (org-publish-cache-ctime-of-src
- (expand-file-name included-file))
- t)))))
- (unless visiting (kill-buffer buf)))
- (if (null pstamp) t
- (let ((ctime (org-publish-cache-ctime-of-src filename)))
- (or (< pstamp ctime)
- (when included-files-ctime
- (not (null (delq nil (mapcar (lambda (ct) (< ctime ct))
- included-files-ctime))))))))))
+ (save-excursion
+ (goto-char (point-min))
+ (while (re-search-forward "^[ \t]*#\\+INCLUDE:" nil t)
+ (let* ((element (org-element-at-point))
+ (included-file
+ (and (eq (org-element-type element) 'keyword)
+ (let ((value (org-element-property :value element)))
+ (and value
+ (string-match "^\\(\".+?\"\\|\\S-+\\)" value)
+ (org-remove-double-quotes
+ (match-string 1 value)))))))
+ (when included-file (add-to-list 'includes (expand-file-name included-file)))))))
+ (unless visiting (kill-buffer buf))
+ (when includes (org-publish-cache-set-file-property filename :includes includes)))
+ ;; return t if needsp or if included files have changed
+ (or needsp
+ (when (delq nil
+ (mapcar (lambda (file)
+ (let ((ct (org-publish-cache-ctime-of-src file)))
+ (and (file-exists-p file)
+ (< ctime ct))))
+ (org-publish-cache-get-file-property filename :includes)))
+ ;; update the timestamp of the published file, but only if buffer is not modified
+ (setq buf (find-file-noselect (expand-file-name filename)))
+ (with-current-buffer buf
+ (when (not (buffer-modified-p)) (set-buffer-modified-p t) (save-buffer)))
+ (unless visiting (kill-buffer buf))
+ t))))
(defun org-publish-cache-set-file-property
(filename property value &optional project-name)
--
2.0.4
next reply other threads:[~2014-08-12 13:09 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-08-12 13:09 Matt Lundin [this message]
2014-08-13 4:32 ` [PATCH] Fix and optimize publishing cache check Matt Lundin
2014-08-14 9:13 ` Nicolas Goaziou
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87oavpyhvn.fsf@fastmail.fm \
--to=mdl@imapmail.org \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.