[PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[Matt Lundin]

* lisp/org.el: (org-entry-properties) Stop scanning for timestamps if
a specific timestamp property (e.g., DEADLINE, SCHEDULED, etc.) is
requested and a match is found. Also, if a specific timestamp property
is requested, do not push non-relevant timestamps onto property list.

This change only effects org-entry-properties when a specific
timestamp is requested with the special flag, as in:

(org-entry-properties nil 'special "SCHEDULED")

Previously, even if only the SCHEDULED timestamp was requested,
org-entry-properties would parse all the timestamps in an entry. This
extra parsing could slow down the construction of agenda views,
especially with entries that contained a large number of log
items (CLOCK, state changes, etc.). The function org-entry-get,
however, is only interested in the first occurrence of the item. When
looking for a specific type of timestamp, org-entry-properties now
stops searching for timestamps after the match is found, unless the
property is "CLOCK".

Here are the relevant ELP results:

Before:

org-entry-get	     296         0.4724579999  0.0015961418
org-entry-properties 31          0.3438769999  0.0110928064

After:

org-entry-get        296         0.1447729999  0.0004890979
org-entry-properties 31          0.015765      0.0005085483
---
 lisp/org.el |   60 ++++++++++++++++++++++++++++++++--------------------------
 1 files changed, 33 insertions(+), 27 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 82c0b46..c4fe6a0 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -1,4 +1,4 @@
-;;; org.el --- Outline-based notes management and organizer
+';;; org.el --- Outline-based notes management and organizer
 ;; Carstens outline-mode for keeping track of everything.
 ;; Copyright (C) 2004, 2005, 2006, 2007, 2008, 2009, 2010
 ;;   Free Software Foundation, Inc.
@@ -13424,32 +13424,38 @@ things up because then unnecessary parsing is avoided."
 		      (member specific
 			      '("SCHEDULED" "DEADLINE" "CLOCK" "CLOSED"
 				"TIMESTAMP" "TIMESTAMP_IA")))
-	      (while (re-search-forward org-maybe-keyword-time-regexp end t)
-		(setq key (if (match-end 1)
-			      (substring (org-match-string-no-properties 1)
-					 0 -1))
-		      string (if (equal key clockstr)
-				 (org-no-properties
-				  (org-trim
-				   (buffer-substring
-				    (match-beginning 3) (goto-char
-							 (point-at-eol)))))
-			       (substring (org-match-string-no-properties 3)
-					  1 -1)))
-		;; Get the correct property name from the key.  This is
-		;; necessary if the user has configured time keywords.
-		(setq key1 (concat key ":"))
-		(cond
-		 ((not key)
-		  (setq key
-			(if (= (char-after (match-beginning 3)) ?\[)
-			    "TIMESTAMP_IA" "TIMESTAMP")))
-		 ((equal key1 org-scheduled-string) (setq key "SCHEDULED"))
-		 ((equal key1 org-deadline-string)  (setq key "DEADLINE"))
-		 ((equal key1 org-closed-string)    (setq key "CLOSED"))
-		 ((equal key1 org-clock-string)     (setq key "CLOCK")))
-		(when (or (equal key "CLOCK") (not (assoc key props)))
-		  (push (cons key string) props))))
+	      (catch 'match
+		(while (re-search-forward org-maybe-keyword-time-regexp end t)
+		  (setq key (if (match-end 1)
+				(substring (org-match-string-no-properties 1)
+					   0 -1))
+			string (if (equal key clockstr)
+				   (org-no-properties
+				    (org-trim
+				     (buffer-substring
+				      (match-beginning 3) (goto-char
+							   (point-at-eol)))))
+				 (substring (org-match-string-no-properties 3)
+					    1 -1)))
+		  ;; Get the correct property name from the key.  This is
+		  ;; necessary if the user has configured time keywords.
+		  (setq key1 (concat key ":"))
+		  (cond
+		   ((not key)
+		    (setq key
+			  (if (= (char-after (match-beginning 3)) ?\[)
+			      "TIMESTAMP_IA" "TIMESTAMP")))
+		   ((equal key1 org-scheduled-string) (setq key "SCHEDULED"))
+		   ((equal key1 org-deadline-string)  (setq key "DEADLINE"))
+		   ((equal key1 org-closed-string)    (setq key "CLOSED"))
+		   ((equal key1 org-clock-string)     (setq key "CLOCK")))
+		  (if (and specific (equal key specific) (not (equal key "CLOCK")))
+		      (progn 
+			(push (cons key string) props)
+			;; no need to search further if match is found
+			(throw 'match t))
+		    (when (or (equal key "CLOCK") (not (assoc key props)))
+		      (push (cons key string) props))))))
 	    )
 
 	  (when (memq which '(all standard))
-- 
1.7.3.3

Re: [PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[Matt Lundin]

A note of clarification: The subject should read patch 1 of 1. 

Thanks,
Matt

Matt Lundin <mdl@imapmail.org> writes:

> * lisp/org.el: (org-entry-properties) Stop scanning for timestamps if
> a specific timestamp property (e.g., DEADLINE, SCHEDULED, etc.) is
> requested and a match is found. Also, if a specific timestamp property
> is requested, do not push non-relevant timestamps onto property list.
>
> This change only effects org-entry-properties when a specific
> timestamp is requested with the special flag, as in:
>
> (org-entry-properties nil 'special "SCHEDULED")
>
> Previously, even if only the SCHEDULED timestamp was requested,
> org-entry-properties would parse all the timestamps in an entry. This
> extra parsing could slow down the construction of agenda views,
> especially with entries that contained a large number of log
> items (CLOCK, state changes, etc.). The function org-entry-get,
> however, is only interested in the first occurrence of the item. When
> looking for a specific type of timestamp, org-entry-properties now
> stops searching for timestamps after the match is found, unless the
> property is "CLOCK".
>
> Here are the relevant ELP results:
>
> Before:
>
> org-entry-get	     296         0.4724579999  0.0015961418
> org-entry-properties 31          0.3438769999  0.0110928064
>
> After:
>
> org-entry-get        296         0.1447729999  0.0004890979
> org-entry-properties 31          0.015765      0.0005085483
> ---
>  lisp/org.el |   60 ++++++++++++++++++++++++++++++++--------------------------
>  1 files changed, 33 insertions(+), 27 deletions(-)
>
> diff --git a/lisp/org.el b/lisp/org.el
> index 82c0b46..c4fe6a0 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -1,4 +1,4 @@
> -;;; org.el --- Outline-based notes management and organizer
> +';;; org.el --- Outline-based notes management and organizer
>  ;; Carstens outline-mode for keeping track of everything.
>  ;; Copyright (C) 2004, 2005, 2006, 2007, 2008, 2009, 2010
>  ;;   Free Software Foundation, Inc.
> @@ -13424,32 +13424,38 @@ things up because then unnecessary parsing is avoided."
>  		      (member specific
>  			      '("SCHEDULED" "DEADLINE" "CLOCK" "CLOSED"
>  				"TIMESTAMP" "TIMESTAMP_IA")))
> -	      (while (re-search-forward org-maybe-keyword-time-regexp end t)
> -		(setq key (if (match-end 1)
> -			      (substring (org-match-string-no-properties 1)
> -					 0 -1))
> -		      string (if (equal key clockstr)
> -				 (org-no-properties
> -				  (org-trim
> -				   (buffer-substring
> -				    (match-beginning 3) (goto-char
> -							 (point-at-eol)))))
> -			       (substring (org-match-string-no-properties 3)
> -					  1 -1)))
> -		;; Get the correct property name from the key.  This is
> -		;; necessary if the user has configured time keywords.
> -		(setq key1 (concat key ":"))
> -		(cond
> -		 ((not key)
> -		  (setq key
> -			(if (= (char-after (match-beginning 3)) ?\[)
> -			    "TIMESTAMP_IA" "TIMESTAMP")))
> -		 ((equal key1 org-scheduled-string) (setq key "SCHEDULED"))
> -		 ((equal key1 org-deadline-string)  (setq key "DEADLINE"))
> -		 ((equal key1 org-closed-string)    (setq key "CLOSED"))
> -		 ((equal key1 org-clock-string)     (setq key "CLOCK")))
> -		(when (or (equal key "CLOCK") (not (assoc key props)))
> -		  (push (cons key string) props))))
> +	      (catch 'match
> +		(while (re-search-forward org-maybe-keyword-time-regexp end t)
> +		  (setq key (if (match-end 1)
> +				(substring (org-match-string-no-properties 1)
> +					   0 -1))
> +			string (if (equal key clockstr)
> +				   (org-no-properties
> +				    (org-trim
> +				     (buffer-substring
> +				      (match-beginning 3) (goto-char
> +							   (point-at-eol)))))
> +				 (substring (org-match-string-no-properties 3)
> +					    1 -1)))
> +		  ;; Get the correct property name from the key.  This is
> +		  ;; necessary if the user has configured time keywords.
> +		  (setq key1 (concat key ":"))
> +		  (cond
> +		   ((not key)
> +		    (setq key
> +			  (if (= (char-after (match-beginning 3)) ?\[)
> +			      "TIMESTAMP_IA" "TIMESTAMP")))
> +		   ((equal key1 org-scheduled-string) (setq key "SCHEDULED"))
> +		   ((equal key1 org-deadline-string)  (setq key "DEADLINE"))
> +		   ((equal key1 org-closed-string)    (setq key "CLOSED"))
> +		   ((equal key1 org-clock-string)     (setq key "CLOCK")))
> +		  (if (and specific (equal key specific) (not (equal key "CLOCK")))
> +		      (progn 
> +			(push (cons key string) props)
> +			;; no need to search further if match is found
> +			(throw 'match t))
> +		    (when (or (equal key "CLOCK") (not (assoc key props)))
> +		      (push (cons key string) props))))))
>  	    )
>  
>  	  (when (memq which '(all standard))

Re: [PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[Matt Lundin]

Matt Lundin <mdl@imapmail.org> writes:

> A note of clarification: The subject should read patch 1 of 1. 
>

One more note. I believe these optimizations are most evident when one
is using org-habit. AFAICT, they do not produce a substantial benefit
when org-habit is not loaded.

Best,
Matt

Re: [PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[Carsten Dominik]

Great work on these optimizations, Matt, thanks!

I guess this one could be faster still by only searching for
that one keyword is `specific' is set.....

Patch applied.

- Carsten

On Dec 13, 2010, at 4:05 AM, Matt Lundin wrote:

> * lisp/org.el: (org-entry-properties) Stop scanning for timestamps if
> a specific timestamp property (e.g., DEADLINE, SCHEDULED, etc.) is
> requested and a match is found. Also, if a specific timestamp property
> is requested, do not push non-relevant timestamps onto property list.
>
> This change only effects org-entry-properties when a specific
> timestamp is requested with the special flag, as in:
>
> (org-entry-properties nil 'special "SCHEDULED")
>
> Previously, even if only the SCHEDULED timestamp was requested,
> org-entry-properties would parse all the timestamps in an entry. This
> extra parsing could slow down the construction of agenda views,
> especially with entries that contained a large number of log
> items (CLOCK, state changes, etc.). The function org-entry-get,
> however, is only interested in the first occurrence of the item. When
> looking for a specific type of timestamp, org-entry-properties now
> stops searching for timestamps after the match is found, unless the
> property is "CLOCK".
>
> Here are the relevant ELP results:
>
> Before:
>
> org-entry-get	     296         0.4724579999  0.0015961418
> org-entry-properties 31          0.3438769999  0.0110928064
>
> After:
>
> org-entry-get        296         0.1447729999  0.0004890979
> org-entry-properties 31          0.015765      0.0005085483
> ---
> lisp/org.el |   60 +++++++++++++++++++++++++++++++ 
> +--------------------------
> 1 files changed, 33 insertions(+), 27 deletions(-)
>
> diff --git a/lisp/org.el b/lisp/org.el
> index 82c0b46..c4fe6a0 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -1,4 +1,4 @@
> -;;; org.el --- Outline-based notes management and organizer
> +';;; org.el --- Outline-based notes management and organizer
> ;; Carstens outline-mode for keeping track of everything.
> ;; Copyright (C) 2004, 2005, 2006, 2007, 2008, 2009, 2010
> ;;   Free Software Foundation, Inc.
> @@ -13424,32 +13424,38 @@ things up because then unnecessary parsing  
> is avoided."
> 		      (member specific
> 			      '("SCHEDULED" "DEADLINE" "CLOCK" "CLOSED"
> 				"TIMESTAMP" "TIMESTAMP_IA")))
> -	      (while (re-search-forward org-maybe-keyword-time-regexp end t)
> -		(setq key (if (match-end 1)
> -			      (substring (org-match-string-no-properties 1)
> -					 0 -1))
> -		      string (if (equal key clockstr)
> -				 (org-no-properties
> -				  (org-trim
> -				   (buffer-substring
> -				    (match-beginning 3) (goto-char
> -							 (point-at-eol)))))
> -			       (substring (org-match-string-no-properties 3)
> -					  1 -1)))
> -		;; Get the correct property name from the key.  This is
> -		;; necessary if the user has configured time keywords.
> -		(setq key1 (concat key ":"))
> -		(cond
> -		 ((not key)
> -		  (setq key
> -			(if (= (char-after (match-beginning 3)) ?\[)
> -			    "TIMESTAMP_IA" "TIMESTAMP")))
> -		 ((equal key1 org-scheduled-string) (setq key "SCHEDULED"))
> -		 ((equal key1 org-deadline-string)  (setq key "DEADLINE"))
> -		 ((equal key1 org-closed-string)    (setq key "CLOSED"))
> -		 ((equal key1 org-clock-string)     (setq key "CLOCK")))
> -		(when (or (equal key "CLOCK") (not (assoc key props)))
> -		  (push (cons key string) props))))
> +	      (catch 'match
> +		(while (re-search-forward org-maybe-keyword-time-regexp end t)
> +		  (setq key (if (match-end 1)
> +				(substring (org-match-string-no-properties 1)
> +					   0 -1))
> +			string (if (equal key clockstr)
> +				   (org-no-properties
> +				    (org-trim
> +				     (buffer-substring
> +				      (match-beginning 3) (goto-char
> +							   (point-at-eol)))))
> +				 (substring (org-match-string-no-properties 3)
> +					    1 -1)))
> +		  ;; Get the correct property name from the key.  This is
> +		  ;; necessary if the user has configured time keywords.
> +		  (setq key1 (concat key ":"))
> +		  (cond
> +		   ((not key)
> +		    (setq key
> +			  (if (= (char-after (match-beginning 3)) ?\[)
> +			      "TIMESTAMP_IA" "TIMESTAMP")))
> +		   ((equal key1 org-scheduled-string) (setq key "SCHEDULED"))
> +		   ((equal key1 org-deadline-string)  (setq key "DEADLINE"))
> +		   ((equal key1 org-closed-string)    (setq key "CLOSED"))
> +		   ((equal key1 org-clock-string)     (setq key "CLOCK")))
> +		  (if (and specific (equal key specific) (not (equal key "CLOCK")))
> +		      (progn
> +			(push (cons key string) props)
> +			;; no need to search further if match is found
> +			(throw 'match t))
> +		    (when (or (equal key "CLOCK") (not (assoc key props)))
> +		      (push (cons key string) props))))))
> 	    )
>
> 	  (when (memq which '(all standard))
> -- 
> 1.7.3.3
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

- Carsten

Re: [PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[Matt Lundin]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> Great work on these optimizations, Matt, thanks!
>
> I guess this one could be faster still by only searching for
> that one keyword is `specific' is set.....
>
> Patch applied.

Thanks! I considered the additional optimization, but being able to use
org-maybe-keyword-time-regexp is so convenient. :) I'll take another
look.

BTW, the two optimization patches do not yet appear in the main git
repository:

http://repo.or.cz/w/org-mode.git

Best,
Matt

Re: [PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[Carsten Dominik]

On Dec 13, 2010, at 1:53 PM, Matt Lundin wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
>
>> Great work on these optimizations, Matt, thanks!
>>
>> I guess this one could be faster still by only searching for
>> that one keyword is `specific' is set.....
>>
>> Patch applied.
>
> Thanks! I considered the additional optimization, but being able to  
> use
> org-maybe-keyword-time-regexp is so convenient. :) I'll take another
> look.
>
> BTW, the two optimization patches do not yet appear in the main git
> repository:
>
> http://repo.or.cz/w/org-mode.git

Should be fixed now - please verify.

- Carsten

Re: [PATCH 2/2] Make timestamp search in org-entry-properties more efficient.

[Matt Lundin]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> On Dec 13, 2010, at 1:53 PM, Matt Lundin wrote:

>> BTW, the two optimization patches do not yet appear in the main git
>> repository:
>>
>> http://repo.or.cz/w/org-mode.git
>
> Should be fixed now - please verify.
>

Yes. Everything is there now. Thanks!

Matt