Estimate ranges in column view

Estimate ranges in column view

[Mike Gauland]

When planning my work, I estimate the effort required as a range, rather than a
single value. That is, instead of estimating a certain task will take 4 days,
I'll use a range of 3-5 days. If I'm a bit less confident I know how long it
will take, I'll use a wider range (e.g., 2-6 days).  

When I first started doing this, I switched from using a single 'Effort' column
in org mode, to two columns (Effort_Low and Effort_High), simply summing each
column to get an estimate for a composite task.  However, this magnifies the
level of uncertainty in the estimate. The final 'Effort_Low' value tells me what
to expect if everything goes optimally; 'Effort_High' provides the extremely
pessimistic view.

More realistic summaries come from considering the range of each pair, using the
combined statistical variance in each (low, high) pair to determine the variance
in the final value. This is the method used by LiquidPlanner, for example.

I've been mucking about with org-colview.el to automate this calculation for me,
and am quite pleased with the results so far. I've approached this by adding a
new summary type ("est") to org-columns-compile-map, and extending
org-columns-number-to-string and org-columns-string-to-number to convert ranges
to and from strings. This lets me populate an 'Estimates' column with values
such as "[2 4]", and specify a summary type "est" to have the algorithm
described above used to produce the final estimates.

I have two questions for the list:

  1. Is this the right approach, or should I change the behaviour of the
existing EFFORT property?
  2. Is this something others would find useful?

Thanks,
Mike

Re: Estimate ranges in column view

[Carsten Dominik]

Hi Mike,

On Jun 17, 2010, at 10:06 PM, Mike Gauland wrote:

> When planning my work, I estimate the effort required as a range,  
> rather than a
> single value. That is, instead of estimating a certain task will  
> take 4 days,
> I'll use a range of 3-5 days. If I'm a bit less confident I know how  
> long it
> will take, I'll use a wider range (e.g., 2-6 days).
>
> When I first started doing this, I switched from using a single  
> 'Effort' column
> in org mode, to two columns (Effort_Low and Effort_High), simply  
> summing each
> column to get an estimate for a composite task.  However, this  
> magnifies the
> level of uncertainty in the estimate. The final 'Effort_Low' value  
> tells me what
> to expect if everything goes optimally; 'Effort_High' provides the  
> extremely
> pessimistic view.
>
> More realistic summaries come from considering the range of each  
> pair, using the
> combined statistical variance in each (low, high) pair to determine  
> the variance
> in the final value. This is the method used by LiquidPlanner, for  
> example.
>
> I've been mucking about with org-colview.el to automate this  
> calculation for me,
> and am quite pleased with the results so far. I've approached this  
> by adding a
> new summary type ("est") to org-columns-compile-map, and extending
> org-columns-number-to-string and org-columns-string-to-number to  
> convert ranges
> to and from strings. This lets me populate an 'Estimates' column  
> with values
> such as "[2 4]", and specify a summary type "est" to have the  
> algorithm
> described above used to produce the final estimates.
>
> I have two questions for the list:
>
>  1. Is this the right approach, or should I change the behaviour of  
> the
> existing EFFORT property?

Changing the existing EFFORT property would require more changes in  
other places for example in the agenda filter that filters by  
estimated effort, or maybe also in  code that helps to set/change the  
effort property.  So to make this fully work with the existing EFFORT  
property would probably require a lot more work.

>  2. Is this something others would find useful?

I like the idea of variance calculation for this purpose.  So I would  
be inclined to take a patch that will introduce this new summary  
operator.

Depending on how much code this is, you'd also have to sign the papers  
with the FSF (unless you have done so for Emacs already).

To complete the patch, you could make things easy for my be providing  
a change to the manual, and by also doing the corresponding changes in  
org-colview-xemacs.el.  But neither of these two would be required for  
acceptance.

Cheers

- Carsten

Re: Estimate ranges in column view

[Michael Gauland]

[-- Attachment #1: Type: Text/Plain, Size: 587 bytes --]

Carsten,
Here is a patch for a new 'est+' summary type, including corresponding changes for xemacs and the manual. I've done basic testing on the GNU emacs version, but not the xemacs code.  

I'm not sure the change to the manual provides the right amount of information
in the right place; I'd be happy to re-write to make it find in
better. Similarly, the name of the summary type is entirely up to you.

I didn't know whether to send this directly to you, or to the list; if it should go to the list I'd be happy to send it there directly.

Thanks for the chance to contribute,
Mike

[-- Attachment #2: estimates-patch --]
[-- Type: Text/Plain, Size: 10145 bytes --]

diff --git a/doc/org.texi b/doc/org.texi
index 17615e0..529be4d 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -4723,9 +4723,10 @@ optional.  The individual parts have the following meaning:
                 @{:min@}    @r{Smallest time value in column.}
                 @{:max@}    @r{Largest time value.}
                 @{:mean@}   @r{Arithmetic mean of time values.}
-                @{@@min@}   @r{Minimum age (in days/hours/mins/seconds).}
-                @{@@max@}   @r{Maximum age (in days/hours/mins/seconds).}
-                @{@@mean@}  @r{Arithmetic mean of ages (in days/hours/mins/seconds).}
+                @{@@min@}    @r{Minimum age (in days/hours/mins/seconds).}
+                @{@@max@}    @r{Maximum age (in days/hours/mins/seconds).}
+                @{@@mean@}   @r{Arithmetic mean of ages (in days/hours/mins/seconds).}
+                @{est+@}    @r{Add low-high estimates.}
 @end example
 
 @noindent
@@ -4733,6 +4734,22 @@ Be aware that you can only have one summary type for any property you
 include. Subsequent columns referencing the same property will all display the
 same summary information.
 
+The 'est+' summary type requires further explanation. It is used for
+combining task estimates, expressed as low-high ranges. For example, instead
+of estimating a particular task will take 5 days, you might estimate it as
+5-6 days if you're fairly confident you know how much woark is required, or
+1-10 days if you don't really know what needs to be done.  Both ranges
+average at 5.5 days, but the first represents a more predictable delivery.
+
+When combining a set of such estimates, simply adding the lows and highs
+produces an unrealistically wide result. Instead, 'est+' adds the statistical
+mean and variance of the sub-tasks, generating a final estimate from the sum.
+For example, suppose you had ten tasks, each of which was estimated at 0.5 to
+2 days of work. Straight addition produces an estimate of 5 to 20 days,
+representing what to expect if everything goes either extremely well or
+extremely poorly. In contrast, 'est+' estimates the full job more
+realistically, at 10-15 days.
+
 Here is an example for a complete columns definition, along with allowed
 values.
 
diff --git a/lisp/org-colview-xemacs.el b/lisp/org-colview-xemacs.el
index 152d9fe..90bf4c4 100644
--- a/lisp/org-colview-xemacs.el
+++ b/lisp/org-colview-xemacs.el
@@ -917,7 +917,8 @@ around it."
     ("@max" max_age max (lambda (x) (- org-columns-time x)))
     ("@mean" mean_age
      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
-     (lambda (x) (- org-columns-time x))))
+     (lambda (x) (- org-columns-time x)))
+    ("est+" estimate org-estimate-combine))
   "Operator <-> format,function,calc  map.
  Used to compile/uncompile columns format and completing read in
  interactive function org-columns-new.
@@ -1206,6 +1207,7 @@ Don't set this, this is meant for dynamic scoping.")
 (defun org-columns-number-to-string (n fmt &optional printf)
   "Convert a computed column number to a string value, according to FMT."
   (cond
+   ((memq fmt '(estimate)) (org-estimate-print n printf))
    ((not (numberp n)) "")
    ((memq fmt '(add_times max_times min_times mean_times))
     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
@@ -1250,9 +1252,9 @@ Don't set this, this is meant for dynamic scoping.")
 	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
 	  sum))
        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
-	(if (equal s "[X]") 1. 0.000001))
-       (t (string-to-number s)))
-    0))
+        (if (equal s "[X]") 1. 0.000001))
+       ((memq fmt '(estimate)) (org-string-to-estimate s))
+       (t (string-to-number s)))))
 
 (defun org-columns-uncompile-format (cfmt)
   "Turn the compiled columns format back into a string representation."
@@ -1693,6 +1695,42 @@ This will add overlays to the date lines, to show the summary for each day."
       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
     ""))
 
+(defun org-estimate-mean-and-var (v)
+  "Return the mean and variance of an estimate."
+  (let* ((low (float (car v)))
+         (high (float (cadr v)))
+         (mean (/ (+ low high) 2.0))
+         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean) 2.0)) 2.0)))
+    (list  mean var)
+    ))
+
+(defun org-estimate-combine (&rest el)
+  "Combine a list of estimates, using mean and variance.
+The mean and variance of the result will be the sum of the means
+and variances (respectively) of the individual estimates."
+  (let ((mean 0)
+        (var 0))
+    (mapc (lambda (e)
+	    (let ((stats (org-estimate-mean-and-var e)))
+	      (setq mean (+ mean (car stats)))
+	      (setq var (+ var (cadr stats)))))
+	  el)
+    (let ((stdev (sqrt var)))
+      (list (- mean stdev) (+ mean stdev)))
+    ))
+
+(defun org-estimate-print (e &optional fmt)
+  "Prepare a string representation of an estimate, as two numbers with a '-' in between them."
+  (if (null fmt) (set 'fmt "%.0f"))
+  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-")))
+
+(defun org-string-to-estimate (s)
+  "Convert a string to an estimate. The string should be two numbers joined with a '-'."
+  (if (string-match "\\(.*\\)-\\(.*\\)" s)
+      (list (string-to-number (match-string 1 s)) (string-to-number(match-string 2 s)))
+    (list (string-to-number s) (string-to-number s))
+    ))
+
 
 (provide 'org-colview)
 (provide 'org-colview-xemacs)
diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index c820be0..af7eef5 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -746,7 +746,8 @@ around it."
     ("@max" max_age max (lambda (x) (- org-columns-time x)))
     ("@mean" mean_age
      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
-     (lambda (x) (- org-columns-time x))))
+     (lambda (x) (- org-columns-time x)))
+    ("est+" estimate org-estimate-combine))
   "Operator <-> format,function,calc  map.
 Used to compile/uncompile columns format and completing read in
 interactive function org-columns-new.
@@ -1031,6 +1032,7 @@ Don't set this, this is meant for dynamic scoping.")
 (defun org-columns-number-to-string (n fmt &optional printf)
   "Convert a computed column number to a string value, according to FMT."
   (cond
+   ((memq fmt '(estimate)) (org-estimate-print n printf))
    ((not (numberp n)) "")
    ((memq fmt '(add_times max_times min_times mean_times))
     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
@@ -1054,28 +1056,30 @@ Don't set this, this is meant for dynamic scoping.")
       (format "[%d/%d]" n m)
     (format "[%d%%]"(floor (+ 0.5 (* 100. (/ (* 1.0 n) m)))))))
 
+
 (defun org-columns-string-to-number (s fmt)
   "Convert a column value to a number that can be used for column computing."
   (if s
       (cond
        ((memq fmt '(min_age max_age mean_age))
-	(cond ((string= s "") org-columns-time)
-	      ((string-match
-		"\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+\\)s"
-		s)
-	       (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match-string 1 s)))
-				    (string-to-number (match-string 2 s))))
-			   (string-to-number (match-string 3 s))))
-		  (string-to-number (match-string 4 s))))
-	      (t (time-to-number-of-days (apply 'encode-time
-						(org-parse-time-string s t))))))
+        (cond ((string= s "") org-columns-time)
+              ((string-match
+                "\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+\\)s"
+                s)
+               (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match-string 1 s)))
+                                    (string-to-number (match-string 2 s))))
+                           (string-to-number (match-string 3 s))))
+                  (string-to-number (match-string 4 s))))
+              (t (time-to-number-of-days (apply 'encode-time
+                                                (org-parse-time-string s t))))))
        ((string-match ":" s)
-	(let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
-	  (while l
-	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
-	  sum))
+        (let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
+          (while l
+            (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
+          sum))
        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
-	(if (equal s "[X]") 1. 0.000001))
+        (if (equal s "[X]") 1. 0.000001))
+       ((memq fmt '(estimate)) (org-string-to-estimate s))
        (t (string-to-number s)))))
 
 (defun org-columns-uncompile-format (cfmt)
@@ -1492,6 +1496,43 @@ This will add overlays to the date lines, to show the summary for each day."
       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
     ""))
 
+(defun org-estimate-mean-and-var (v)
+  "Return the mean and variance of an estimate."
+  (let* ((low (float (car v)))
+         (high (float (cadr v)))
+         (mean (/ (+ low high) 2.0))
+         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean) 2.0)) 2.0)))
+    (list  mean var)
+    ))
+
+(defun org-estimate-combine (&rest el)
+  "Combine a list of estimates, using mean and variance.
+The mean and variance of the result will be the sum of the means
+and variances (respectively) of the individual estimates."
+  (let ((mean 0)
+        (var 0))
+    (mapc (lambda (e)
+              (let ((stats (org-estimate-mean-and-var e)))
+                (setq mean (+ mean (car stats)))
+                (setq var (+ var (cadr stats)))))
+            el)
+    (let ((stdev (sqrt var)))
+      (list (- mean stdev) (+ mean stdev)))
+    ))
+
+(defun org-estimate-print (e &optional fmt)
+  "Prepare a string representation of an estimate, as two numbers with a '-' in between them."
+  (if (null fmt) (set 'fmt "%.0f"))
+  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-"))
+  )
+
+(defun org-string-to-estimate (s)
+  "Convert a string to an estimate. The string should be two numbers joined with a '-'."
+  (if (string-match "\\(.*\\)-\\(.*\\)" s)
+      (list (string-to-number (match-string 1 s)) (string-to-number(match-string 2 s)))
+    (list (string-to-number s) (string-to-number s))
+    ))
+
 
 (provide 'org-colview)
 

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Estimate ranges in column view

[Carsten Dominik]

Hi Michael,

I was about to apply the patch, but I don't think I got a reply from  
you regarding copyright issues.
Did I miss that, or did you not say anything about it?

Unfortunately the patch is larger than the maximum that I can take  
without copyright assignment.



- Carsten

On Jun 22, 2010, at 4:36 AM, Michael Gauland wrote:

> Carsten,
> Here is a patch for a new 'est+' summary type, including  
> corresponding changes for xemacs and the manual. I've done basic  
> testing on the GNU emacs version, but not the xemacs code.
>
> I'm not sure the change to the manual provides the right amount of  
> information
> in the right place; I'd be happy to re-write to make it find in
> better. Similarly, the name of the summary type is entirely up to you.
>
> I didn't know whether to send this directly to you, or to the list;  
> if it should go to the list I'd be happy to send it there directly.
>
> Thanks for the chance to contribute,
> Mike
> diff --git a/doc/org.texi b/doc/org.texi
> index 17615e0..529be4d 100644
> --- a/doc/org.texi
> +++ b/doc/org.texi
> @@ -4723,9 +4723,10 @@ optional.  The individual parts have the  
> following meaning:
>                 @{:min@}    @r{Smallest time value in column.}
>                 @{:max@}    @r{Largest time value.}
>                 @{:mean@}   @r{Arithmetic mean of time values.}
> -                @{@@min@}   @r{Minimum age (in days/hours/mins/ 
> seconds).}
> -                @{@@max@}   @r{Maximum age (in days/hours/mins/ 
> seconds).}
> -                @{@@mean@}  @r{Arithmetic mean of ages (in days/ 
> hours/mins/seconds).}
> +                @{@@min@}    @r{Minimum age (in days/hours/mins/ 
> seconds).}
> +                @{@@max@}    @r{Maximum age (in days/hours/mins/ 
> seconds).}
> +                @{@@mean@}   @r{Arithmetic mean of ages (in days/ 
> hours/mins/seconds).}
> +                @{est+@}    @r{Add low-high estimates.}
> @end example
>
> @noindent
> @@ -4733,6 +4734,22 @@ Be aware that you can only have one summary  
> type for any property you
> include. Subsequent columns referencing the same property will all  
> display the
> same summary information.
>
> +The 'est+' summary type requires further explanation. It is used for
> +combining task estimates, expressed as low-high ranges. For  
> example, instead
> +of estimating a particular task will take 5 days, you might  
> estimate it as
> +5-6 days if you're fairly confident you know how much woark is  
> required, or
> +1-10 days if you don't really know what needs to be done.  Both  
> ranges
> +average at 5.5 days, but the first represents a more predictable  
> delivery.
> +
> +When combining a set of such estimates, simply adding the lows and  
> highs
> +produces an unrealistically wide result. Instead, 'est+' adds the  
> statistical
> +mean and variance of the sub-tasks, generating a final estimate  
> from the sum.
> +For example, suppose you had ten tasks, each of which was estimated  
> at 0.5 to
> +2 days of work. Straight addition produces an estimate of 5 to 20  
> days,
> +representing what to expect if everything goes either extremely  
> well or
> +extremely poorly. In contrast, 'est+' estimates the full job more
> +realistically, at 10-15 days.
> +
> Here is an example for a complete columns definition, along with  
> allowed
> values.
>
> diff --git a/lisp/org-colview-xemacs.el b/lisp/org-colview-xemacs.el
> index 152d9fe..90bf4c4 100644
> --- a/lisp/org-colview-xemacs.el
> +++ b/lisp/org-colview-xemacs.el
> @@ -917,7 +917,8 @@ around it."
>     ("@max" max_age max (lambda (x) (- org-columns-time x)))
>     ("@mean" mean_age
>      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
> -     (lambda (x) (- org-columns-time x))))
> +     (lambda (x) (- org-columns-time x)))
> +    ("est+" estimate org-estimate-combine))
>   "Operator <-> format,function,calc  map.
>  Used to compile/uncompile columns format and completing read in
>  interactive function org-columns-new.
> @@ -1206,6 +1207,7 @@ Don't set this, this is meant for dynamic  
> scoping.")
> (defun org-columns-number-to-string (n fmt &optional printf)
>   "Convert a computed column number to a string value, according to  
> FMT."
>   (cond
> +   ((memq fmt '(estimate)) (org-estimate-print n printf))
>    ((not (numberp n)) "")
>    ((memq fmt '(add_times max_times min_times mean_times))
>     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
> @@ -1250,9 +1252,9 @@ Don't set this, this is meant for dynamic  
> scoping.")
> 	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> 	  sum))
>        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
> -	(if (equal s "[X]") 1. 0.000001))
> -       (t (string-to-number s)))
> -    0))
> +        (if (equal s "[X]") 1. 0.000001))
> +       ((memq fmt '(estimate)) (org-string-to-estimate s))
> +       (t (string-to-number s)))))
>
> (defun org-columns-uncompile-format (cfmt)
>   "Turn the compiled columns format back into a string  
> representation."
> @@ -1693,6 +1695,42 @@ This will add overlays to the date lines, to  
> show the summary for each day."
>       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
>     ""))
>
> +(defun org-estimate-mean-and-var (v)
> +  "Return the mean and variance of an estimate."
> +  (let* ((low (float (car v)))
> +         (high (float (cadr v)))
> +         (mean (/ (+ low high) 2.0))
> +         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean)  
> 2.0)) 2.0)))
> +    (list  mean var)
> +    ))
> +
> +(defun org-estimate-combine (&rest el)
> +  "Combine a list of estimates, using mean and variance.
> +The mean and variance of the result will be the sum of the means
> +and variances (respectively) of the individual estimates."
> +  (let ((mean 0)
> +        (var 0))
> +    (mapc (lambda (e)
> +	    (let ((stats (org-estimate-mean-and-var e)))
> +	      (setq mean (+ mean (car stats)))
> +	      (setq var (+ var (cadr stats)))))
> +	  el)
> +    (let ((stdev (sqrt var)))
> +      (list (- mean stdev) (+ mean stdev)))
> +    ))
> +
> +(defun org-estimate-print (e &optional fmt)
> +  "Prepare a string representation of an estimate, as two numbers  
> with a '-' in between them."
> +  (if (null fmt) (set 'fmt "%.0f"))
> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-")))
> +
> +(defun org-string-to-estimate (s)
> +  "Convert a string to an estimate. The string should be two  
> numbers joined with a '-'."
> +  (if (string-match "\\(.*\\)-\\(.*\\)" s)
> +      (list (string-to-number (match-string 1 s)) (string-to- 
> number(match-string 2 s)))
> +    (list (string-to-number s) (string-to-number s))
> +    ))
> +
>
> (provide 'org-colview)
> (provide 'org-colview-xemacs)
> diff --git a/lisp/org-colview.el b/lisp/org-colview.el
> index c820be0..af7eef5 100644
> --- a/lisp/org-colview.el
> +++ b/lisp/org-colview.el
> @@ -746,7 +746,8 @@ around it."
>     ("@max" max_age max (lambda (x) (- org-columns-time x)))
>     ("@mean" mean_age
>      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
> -     (lambda (x) (- org-columns-time x))))
> +     (lambda (x) (- org-columns-time x)))
> +    ("est+" estimate org-estimate-combine))
>   "Operator <-> format,function,calc  map.
> Used to compile/uncompile columns format and completing read in
> interactive function org-columns-new.
> @@ -1031,6 +1032,7 @@ Don't set this, this is meant for dynamic  
> scoping.")
> (defun org-columns-number-to-string (n fmt &optional printf)
>   "Convert a computed column number to a string value, according to  
> FMT."
>   (cond
> +   ((memq fmt '(estimate)) (org-estimate-print n printf))
>    ((not (numberp n)) "")
>    ((memq fmt '(add_times max_times min_times mean_times))
>     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
> @@ -1054,28 +1056,30 @@ Don't set this, this is meant for dynamic  
> scoping.")
>       (format "[%d/%d]" n m)
>     (format "[%d%%]"(floor (+ 0.5 (* 100. (/ (* 1.0 n) m)))))))
>
> +
> (defun org-columns-string-to-number (s fmt)
>   "Convert a column value to a number that can be used for column  
> computing."
>   (if s
>       (cond
>        ((memq fmt '(min_age max_age mean_age))
> -	(cond ((string= s "") org-columns-time)
> -	      ((string-match
> -		"\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+\\)s"
> -		s)
> -	       (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match-string  
> 1 s)))
> -				    (string-to-number (match-string 2 s))))
> -			   (string-to-number (match-string 3 s))))
> -		  (string-to-number (match-string 4 s))))
> -	      (t (time-to-number-of-days (apply 'encode-time
> -						(org-parse-time-string s t))))))
> +        (cond ((string= s "") org-columns-time)
> +              ((string-match
> +                "\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+ 
> \\)s"
> +                s)
> +               (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match- 
> string 1 s)))
> +                                    (string-to-number (match-string  
> 2 s))))
> +                           (string-to-number (match-string 3 s))))
> +                  (string-to-number (match-string 4 s))))
> +              (t (time-to-number-of-days (apply 'encode-time
> +                                                (org-parse-time- 
> string s t))))))
>        ((string-match ":" s)
> -	(let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
> -	  (while l
> -	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> -	  sum))
> +        (let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
> +          (while l
> +            (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> +          sum))
>        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
> -	(if (equal s "[X]") 1. 0.000001))
> +        (if (equal s "[X]") 1. 0.000001))
> +       ((memq fmt '(estimate)) (org-string-to-estimate s))
>        (t (string-to-number s)))))
>
> (defun org-columns-uncompile-format (cfmt)
> @@ -1492,6 +1496,43 @@ This will add overlays to the date lines, to  
> show the summary for each day."
>       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
>     ""))
>
> +(defun org-estimate-mean-and-var (v)
> +  "Return the mean and variance of an estimate."
> +  (let* ((low (float (car v)))
> +         (high (float (cadr v)))
> +         (mean (/ (+ low high) 2.0))
> +         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean)  
> 2.0)) 2.0)))
> +    (list  mean var)
> +    ))
> +
> +(defun org-estimate-combine (&rest el)
> +  "Combine a list of estimates, using mean and variance.
> +The mean and variance of the result will be the sum of the means
> +and variances (respectively) of the individual estimates."
> +  (let ((mean 0)
> +        (var 0))
> +    (mapc (lambda (e)
> +              (let ((stats (org-estimate-mean-and-var e)))
> +                (setq mean (+ mean (car stats)))
> +                (setq var (+ var (cadr stats)))))
> +            el)
> +    (let ((stdev (sqrt var)))
> +      (list (- mean stdev) (+ mean stdev)))
> +    ))
> +
> +(defun org-estimate-print (e &optional fmt)
> +  "Prepare a string representation of an estimate, as two numbers  
> with a '-' in between them."
> +  (if (null fmt) (set 'fmt "%.0f"))
> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-"))
> +  )
> +
> +(defun org-string-to-estimate (s)
> +  "Convert a string to an estimate. The string should be two  
> numbers joined with a '-'."
> +  (if (string-match "\\(.*\\)-\\(.*\\)" s)
> +      (list (string-to-number (match-string 1 s)) (string-to- 
> number(match-string 2 s)))
> +    (list (string-to-number s) (string-to-number s))
> +    ))
> +
>
> (provide 'org-colview)
>

- Carsten

Re: Estimate ranges in column view

[Michael Gauland]

Carsten,

I send the from to assign@gnu.org just before I sent the patch, but I
haven't received a reply. Should I expect one, or is that all I need to do?

Thanks,
Mike

Re: Estimate ranges in column view

[Juan]

On Thu, Jul 01, 2010 at 06:19:02AM +1200, Michael Gauland wrote:
> I send the from to assign@gnu.org just before I sent the patch, but I
> haven't received a reply. Should I expect one, or is that all I need to do?

It goes like this:
 - submit form online
 - wait some days
 - receive a contract to sign (with an ugly FSF sticker for your
   laptop)
 - sign contract, mail back

Then nothing happens (at least in my case).

But someone at FSF should also sign the contract, and you should get
some feedback.

Regards,
Juan Pechiar

Re: Estimate ranges in column view

[Carsten Dominik]

On Jun 30, 2010, at 8:19 PM, Michael Gauland wrote:

> Carsten,
>
> I send the from to assign@gnu.org just before I sent the patch, but I
> haven't received a reply. Should I expect one, or is that all I need  
> to do?

If you have not gotten a reply, please write to them again and ask  
about it, CC to me.

Thanks.

>
> Thanks,
> Mike

- Carsten

Patchwork: Patch 65 Accepted

[Carsten Dominik]

Patch 65 (http://patchwork.newartisans.com/patch/65/) is now Accepted.

This relates to the following submission:

http://mid.gmane.org/%3C20100622.143610.09007983.mikelygee%40no8wireless.co.nz%3E

Re: Estimate ranges in column view

[Carsten Dominik]

Michael,

I have applied the patch - please check it if it is still working  
correctly.

Thanks!

- Carsten

On Jun 22, 2010, at 4:36 AM, Michael Gauland wrote:

> Carsten,
> Here is a patch for a new 'est+' summary type, including  
> corresponding changes for xemacs and the manual. I've done basic  
> testing on the GNU emacs version, but not the xemacs code.
>
> I'm not sure the change to the manual provides the right amount of  
> information
> in the right place; I'd be happy to re-write to make it find in
> better. Similarly, the name of the summary type is entirely up to you.
>
> I didn't know whether to send this directly to you, or to the list;  
> if it should go to the list I'd be happy to send it there directly.
>
> Thanks for the chance to contribute,
> Mike
> diff --git a/doc/org.texi b/doc/org.texi
> index 17615e0..529be4d 100644
> --- a/doc/org.texi
> +++ b/doc/org.texi
> @@ -4723,9 +4723,10 @@ optional.  The individual parts have the  
> following meaning:
>                 @{:min@}    @r{Smallest time value in column.}
>                 @{:max@}    @r{Largest time value.}
>                 @{:mean@}   @r{Arithmetic mean of time values.}
> -                @{@@min@}   @r{Minimum age (in days/hours/mins/ 
> seconds).}
> -                @{@@max@}   @r{Maximum age (in days/hours/mins/ 
> seconds).}
> -                @{@@mean@}  @r{Arithmetic mean of ages (in days/ 
> hours/mins/seconds).}
> +                @{@@min@}    @r{Minimum age (in days/hours/mins/ 
> seconds).}
> +                @{@@max@}    @r{Maximum age (in days/hours/mins/ 
> seconds).}
> +                @{@@mean@}   @r{Arithmetic mean of ages (in days/ 
> hours/mins/seconds).}
> +                @{est+@}    @r{Add low-high estimates.}
> @end example
>
> @noindent
> @@ -4733,6 +4734,22 @@ Be aware that you can only have one summary  
> type for any property you
> include. Subsequent columns referencing the same property will all  
> display the
> same summary information.
>
> +The 'est+' summary type requires further explanation. It is used for
> +combining task estimates, expressed as low-high ranges. For  
> example, instead
> +of estimating a particular task will take 5 days, you might  
> estimate it as
> +5-6 days if you're fairly confident you know how much woark is  
> required, or
> +1-10 days if you don't really know what needs to be done.  Both  
> ranges
> +average at 5.5 days, but the first represents a more predictable  
> delivery.
> +
> +When combining a set of such estimates, simply adding the lows and  
> highs
> +produces an unrealistically wide result. Instead, 'est+' adds the  
> statistical
> +mean and variance of the sub-tasks, generating a final estimate  
> from the sum.
> +For example, suppose you had ten tasks, each of which was estimated  
> at 0.5 to
> +2 days of work. Straight addition produces an estimate of 5 to 20  
> days,
> +representing what to expect if everything goes either extremely  
> well or
> +extremely poorly. In contrast, 'est+' estimates the full job more
> +realistically, at 10-15 days.
> +
> Here is an example for a complete columns definition, along with  
> allowed
> values.
>
> diff --git a/lisp/org-colview-xemacs.el b/lisp/org-colview-xemacs.el
> index 152d9fe..90bf4c4 100644
> --- a/lisp/org-colview-xemacs.el
> +++ b/lisp/org-colview-xemacs.el
> @@ -917,7 +917,8 @@ around it."
>     ("@max" max_age max (lambda (x) (- org-columns-time x)))
>     ("@mean" mean_age
>      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
> -     (lambda (x) (- org-columns-time x))))
> +     (lambda (x) (- org-columns-time x)))
> +    ("est+" estimate org-estimate-combine))
>   "Operator <-> format,function,calc  map.
>  Used to compile/uncompile columns format and completing read in
>  interactive function org-columns-new.
> @@ -1206,6 +1207,7 @@ Don't set this, this is meant for dynamic  
> scoping.")
> (defun org-columns-number-to-string (n fmt &optional printf)
>   "Convert a computed column number to a string value, according to  
> FMT."
>   (cond
> +   ((memq fmt '(estimate)) (org-estimate-print n printf))
>    ((not (numberp n)) "")
>    ((memq fmt '(add_times max_times min_times mean_times))
>     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
> @@ -1250,9 +1252,9 @@ Don't set this, this is meant for dynamic  
> scoping.")
> 	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> 	  sum))
>        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
> -	(if (equal s "[X]") 1. 0.000001))
> -       (t (string-to-number s)))
> -    0))
> +        (if (equal s "[X]") 1. 0.000001))
> +       ((memq fmt '(estimate)) (org-string-to-estimate s))
> +       (t (string-to-number s)))))
>
> (defun org-columns-uncompile-format (cfmt)
>   "Turn the compiled columns format back into a string  
> representation."
> @@ -1693,6 +1695,42 @@ This will add overlays to the date lines, to  
> show the summary for each day."
>       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
>     ""))
>
> +(defun org-estimate-mean-and-var (v)
> +  "Return the mean and variance of an estimate."
> +  (let* ((low (float (car v)))
> +         (high (float (cadr v)))
> +         (mean (/ (+ low high) 2.0))
> +         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean)  
> 2.0)) 2.0)))
> +    (list  mean var)
> +    ))
> +
> +(defun org-estimate-combine (&rest el)
> +  "Combine a list of estimates, using mean and variance.
> +The mean and variance of the result will be the sum of the means
> +and variances (respectively) of the individual estimates."
> +  (let ((mean 0)
> +        (var 0))
> +    (mapc (lambda (e)
> +	    (let ((stats (org-estimate-mean-and-var e)))
> +	      (setq mean (+ mean (car stats)))
> +	      (setq var (+ var (cadr stats)))))
> +	  el)
> +    (let ((stdev (sqrt var)))
> +      (list (- mean stdev) (+ mean stdev)))
> +    ))
> +
> +(defun org-estimate-print (e &optional fmt)
> +  "Prepare a string representation of an estimate, as two numbers  
> with a '-' in between them."
> +  (if (null fmt) (set 'fmt "%.0f"))
> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-")))
> +
> +(defun org-string-to-estimate (s)
> +  "Convert a string to an estimate. The string should be two  
> numbers joined with a '-'."
> +  (if (string-match "\\(.*\\)-\\(.*\\)" s)
> +      (list (string-to-number (match-string 1 s)) (string-to- 
> number(match-string 2 s)))
> +    (list (string-to-number s) (string-to-number s))
> +    ))
> +
>
> (provide 'org-colview)
> (provide 'org-colview-xemacs)
> diff --git a/lisp/org-colview.el b/lisp/org-colview.el
> index c820be0..af7eef5 100644
> --- a/lisp/org-colview.el
> +++ b/lisp/org-colview.el
> @@ -746,7 +746,8 @@ around it."
>     ("@max" max_age max (lambda (x) (- org-columns-time x)))
>     ("@mean" mean_age
>      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
> -     (lambda (x) (- org-columns-time x))))
> +     (lambda (x) (- org-columns-time x)))
> +    ("est+" estimate org-estimate-combine))
>   "Operator <-> format,function,calc  map.
> Used to compile/uncompile columns format and completing read in
> interactive function org-columns-new.
> @@ -1031,6 +1032,7 @@ Don't set this, this is meant for dynamic  
> scoping.")
> (defun org-columns-number-to-string (n fmt &optional printf)
>   "Convert a computed column number to a string value, according to  
> FMT."
>   (cond
> +   ((memq fmt '(estimate)) (org-estimate-print n printf))
>    ((not (numberp n)) "")
>    ((memq fmt '(add_times max_times min_times mean_times))
>     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
> @@ -1054,28 +1056,30 @@ Don't set this, this is meant for dynamic  
> scoping.")
>       (format "[%d/%d]" n m)
>     (format "[%d%%]"(floor (+ 0.5 (* 100. (/ (* 1.0 n) m)))))))
>
> +
> (defun org-columns-string-to-number (s fmt)
>   "Convert a column value to a number that can be used for column  
> computing."
>   (if s
>       (cond
>        ((memq fmt '(min_age max_age mean_age))
> -	(cond ((string= s "") org-columns-time)
> -	      ((string-match
> -		"\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+\\)s"
> -		s)
> -	       (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match-string  
> 1 s)))
> -				    (string-to-number (match-string 2 s))))
> -			   (string-to-number (match-string 3 s))))
> -		  (string-to-number (match-string 4 s))))
> -	      (t (time-to-number-of-days (apply 'encode-time
> -						(org-parse-time-string s t))))))
> +        (cond ((string= s "") org-columns-time)
> +              ((string-match
> +                "\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+ 
> \\)s"
> +                s)
> +               (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match- 
> string 1 s)))
> +                                    (string-to-number (match-string  
> 2 s))))
> +                           (string-to-number (match-string 3 s))))
> +                  (string-to-number (match-string 4 s))))
> +              (t (time-to-number-of-days (apply 'encode-time
> +                                                (org-parse-time- 
> string s t))))))
>        ((string-match ":" s)
> -	(let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
> -	  (while l
> -	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> -	  sum))
> +        (let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
> +          (while l
> +            (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> +          sum))
>        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
> -	(if (equal s "[X]") 1. 0.000001))
> +        (if (equal s "[X]") 1. 0.000001))
> +       ((memq fmt '(estimate)) (org-string-to-estimate s))
>        (t (string-to-number s)))))
>
> (defun org-columns-uncompile-format (cfmt)
> @@ -1492,6 +1496,43 @@ This will add overlays to the date lines, to  
> show the summary for each day."
>       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
>     ""))
>
> +(defun org-estimate-mean-and-var (v)
> +  "Return the mean and variance of an estimate."
> +  (let* ((low (float (car v)))
> +         (high (float (cadr v)))
> +         (mean (/ (+ low high) 2.0))
> +         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean)  
> 2.0)) 2.0)))
> +    (list  mean var)
> +    ))
> +
> +(defun org-estimate-combine (&rest el)
> +  "Combine a list of estimates, using mean and variance.
> +The mean and variance of the result will be the sum of the means
> +and variances (respectively) of the individual estimates."
> +  (let ((mean 0)
> +        (var 0))
> +    (mapc (lambda (e)
> +              (let ((stats (org-estimate-mean-and-var e)))
> +                (setq mean (+ mean (car stats)))
> +                (setq var (+ var (cadr stats)))))
> +            el)
> +    (let ((stdev (sqrt var)))
> +      (list (- mean stdev) (+ mean stdev)))
> +    ))
> +
> +(defun org-estimate-print (e &optional fmt)
> +  "Prepare a string representation of an estimate, as two numbers  
> with a '-' in between them."
> +  (if (null fmt) (set 'fmt "%.0f"))
> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-"))
> +  )
> +
> +(defun org-string-to-estimate (s)
> +  "Convert a string to an estimate. The string should be two  
> numbers joined with a '-'."
> +  (if (string-match "\\(.*\\)-\\(.*\\)" s)
> +      (list (string-to-number (match-string 1 s)) (string-to- 
> number(match-string 2 s)))
> +    (list (string-to-number s) (string-to-number s))
> +    ))
> +
>
> (provide 'org-colview)
>

- Carsten

Re: Estimate ranges in column view

[Michael Gauland]

Great news, Carsten--still works correctly.

Thanks for the chance to contribute!

--Mike

Re: Estimate ranges in column view

[Sebastien Vauban]

Hi Carsten and Michael,

Carsten Dominik wrote:
> On Jun 22, 2010, at 4:36 AM, Michael Gauland wrote:
>> Here is a patch for a new 'est+' summary type, including corresponding
>> changes for xemacs and the manual. I've done basic testing on the GNU emacs
>> version, but not the xemacs code.
>>
>> [...]
>>
>> +                @{est+@}    @r{Add low-high estimates.}
>>
>> [...]
>>
>> +For example, suppose you had ten tasks, each of which was estimated at 0.5
>> +to 2 days of work. Straight addition produces an estimate of 5 to 20 days,
>> +representing what to expect if everything goes either extremely well or
>> +extremely poorly. In contrast, 'est+' estimates the full job more
>> +realistically, at 10-15 days.

Though, if we take 2 tasks, with an estimation of:

- exactly 1 day for task 1, and
- between 0.5 day (min) and 0.75 day (max) for task 2,

we get an estimation of 2 days for both together...

#+BEGIN: columnview :hlines 1 :maxlevel 2
| Task          |   Estim. |
|---------------+----------|
| * Development |      2-2 |
| ** Task 1     |        1 |
| ** Task 2     | 0.5-0.75 |
#+END:

>> +(defun org-estimate-print (e &optional fmt)
>> +  "Prepare a string representation of an estimate, as two numbers with a
>> -' in between them."
>> +  (if (null fmt) (set 'fmt "%.0f"))
>> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-")))

That's because of the rounding to the closest integer, done in the above
function.

Shouldn't we allow for 1 or 2 decimals?

Or is there a possible way to guess what the best rounding could be?

>> (provide 'org-colview)
>
> I have applied the patch - please check it if it is still working correctly.
>
> - Carsten

Best regards,
  Seb

-- 
Sebastien Vauban

Re: Estimate ranges in column view

[Sebastien Vauban]

Hello,

"Sebastien Vauban" wrote:
> Carsten Dominik wrote:
>> On Jun 22, 2010, at 4:36 AM, Michael Gauland wrote:
>>> Here is a patch for a new 'est+' summary type, including corresponding
>>> changes for xemacs and the manual. I've done basic testing on the GNU emacs
>>> version, but not the xemacs code.
>>>
>>> [...]
>>>
>>> +                @{est+@}    @r{Add low-high estimates.}
>>>
>>> [...]
>>>
>>> +For example, suppose you had ten tasks, each of which was estimated at 0.5
>>> +to 2 days of work. Straight addition produces an estimate of 5 to 20 days,
>>> +representing what to expect if everything goes either extremely well or
>>> +extremely poorly. In contrast, 'est+' estimates the full job more
>>> +realistically, at 10-15 days.
>
> Though, if we take 2 tasks, with an estimation of:
>
> - exactly 1 day for task 1, and
> - between 0.5 day (min) and 0.75 day (max) for task 2,
>
> we get an estimation of 2 days for both together...
>
> #+BEGIN: columnview :hlines 1 :maxlevel 2
> | Task          |   Estim. |
> |---------------+----------|
> | * Development |      2-2 |
> | ** Task 1     |        1 |
> | ** Task 2     | 0.5-0.75 |
> #+END:
>
>>> +(defun org-estimate-print (e &optional fmt)
>>> +  "Prepare a string representation of an estimate, as two numbers with a
>>> -' in between them."
>>> +  (if (null fmt) (set 'fmt "%.0f"))
>>> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-")))
>
> That's because of the rounding to the closest integer, done in the above
> function.
>
> Shouldn't we allow for 1 or 2 decimals?
>
> Or is there a possible way to guess what the best rounding could be?

Another note: wouldn't it better to just write "2" when both bounds (min and
max) are equal, that is *not* displaying an estimate of "2-2"?

Best regards,
  Seb

-- 
Sebastien Vauban