Adding support for limits and exclusions to iCal recurring events

Adding support for limits and exclusions to iCal recurring events

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 2668 bytes --]

Hi

I've been wanting to have support for exporting recurring events to iCal
while limiting the number of repetitions, as well as excluding dates.
The patch below is my attempt to add this functionality. However, I'm in
doubt as to whether or not this is the right way to go about it (parsing
dates stored in the property drawer), or if there's a better way?

A sample headline supporting this might look like this:

* Test
:PROPERTIES:
:END_DATE: [2013-07-21 Sun]
:EXCLUDE:  [2013-07-14 Sun]
:ID:       b376f8e3-f1a7-4ed6-ab9b-a255938af8c0
:END:
<2013-07-07 Sun +1w>


If this is a reasonable way to go about it, I'll be happy to resubmit it
as a proper patch (i.e. from `git format-patch`) for inclusion. :)

-Toke

diff --git a/lisp/ox-icalendar.el b/lisp/ox-icalendar.el
index c6ab295..361cef2 100644
--- a/lisp/ox-icalendar.el
+++ b/lisp/ox-icalendar.el
@@ -632,6 +632,12 @@ inlinetask within the section."
        ;; Don't forget components from inner entries.
        contents))))
 
+(defun org-icalendar-zero-convert-timestamp (timestamp &optional format)
+  "Parse and format an org-formatted timestamp, zeroing the time component."
+  (let ((time (org-parse-time-string timestamp))
+	(fmt (or format "%Y%m%dT%H%M%SZ")))
+    (format-time-string fmt (encode-time 0 0 0 (nth 3 time) (nth 4 time) (nth 5 time)))))
+
 (defun org-icalendar--vevent
   (entry timestamp uid summary location description categories)
   "Create a VEVENT component.
@@ -656,11 +662,18 @@ Return VEVENT component as a string."
 	     (org-icalendar-convert-timestamp timestamp "DTEND" t) "\n"
 	     ;; RRULE.
 	     (when (org-element-property :repeater-type timestamp)
-	       (format "RRULE:FREQ=%s;INTERVAL=%d\n"
+	       (format "RRULE:FREQ=%s;INTERVAL=%d%s\n"
 		       (case (org-element-property :repeater-unit timestamp)
 			 (hour "HOURLY") (day "DAILY") (week "WEEKLY")
 			 (month "MONTHLY") (year "YEARLY"))
-		       (org-element-property :repeater-value timestamp)))
+		       (org-element-property :repeater-value timestamp)
+		       (if (not (org-element-property :END_DATE entry)) ""
+			   (format ";UNTIL=%s" (org-icalendar-zero-convert-timestamp
+						  (org-element-property :END_DATE entry))))))
+	     (when (org-element-property :EXCLUDE entry)
+	       (format "EXDATE;VALUE=DATE:%s\n" (mapconcat (lambda (ts) (org-icalendar-zero-convert-timestamp ts "%Y%m%d"))
+							   (split-string (org-element-property :EXCLUDE entry) ",")
+							   ",")))
 	     "SUMMARY:" summary "\n"
 	     (and (org-string-nw-p location) (format "LOCATION:%s\n" location))
 	     (and (org-string-nw-p description)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: Adding support for limits and exclusions to iCal recurring events

[Nicolas Goaziou]

Hello,

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> I've been wanting to have support for exporting recurring events to iCal
> while limiting the number of repetitions, as well as excluding dates.
> The patch below is my attempt to add this functionality. However, I'm in
> doubt as to whether or not this is the right way to go about it (parsing
> dates stored in the property drawer), or if there's a better way?
>
> A sample headline supporting this might look like this:
>
> * Test
> :PROPERTIES:
> :END_DATE: [2013-07-21 Sun]
> :EXCLUDE:  [2013-07-14 Sun]
> :ID:       b376f8e3-f1a7-4ed6-ab9b-a255938af8c0
> :END:
> <2013-07-07 Sun +1w>
>
>
> If this is a reasonable way to go about it, I'll be happy to resubmit it
> as a proper patch (i.e. from `git format-patch`) for inclusion. :)

Thanks for your patch.

I think EXCLUDE property is a good idea, but I may be renamed to
ICALENDAR_EXCLUDE until it is also handled by Org Agenda, if it ever
happens. It also needs to be documented in the manual.

On the other hand, I'm not sure about the END_DATE property. Couldn't
DEADLINE be used for that matter? For example, `repeater-end-date' could
be added to `org-icalendar-use-deadline' possible values. When this
symbol belongs to variable's value and current entry has a deadline, any
timestamps with a repeater get deadline's value as its END DATE
property.

What do you think?

> +(defun org-icalendar-zero-convert-timestamp (timestamp &optional format)
> +  "Parse and format an org-formatted timestamp, zeroing the time component."
> +  (let ((time (org-parse-time-string timestamp))
> +	(fmt (or format "%Y%m%dT%H%M%SZ")))
> +    (format-time-string fmt (encode-time 0 0 0 (nth 3 time) (nth 4 time) (nth 5 time)))))

Can't `org-icalendar-convert-timestamp' be used instead?


Regards,

-- 
Nicolas Goaziou

Re: Adding support for limits and exclusions to iCal recurring events

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 3729 bytes --]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I think EXCLUDE property is a good idea, but I may be renamed to
> ICALENDAR_EXCLUDE until it is also handled by Org Agenda, if it ever
> happens. It also needs to be documented in the manual.

Handling it in the agenda can be done by an exclusion function. I
currently have this in my org-agenda-skip-function-global (along with
another function to skip items after their END_DATE):

(defun thj/skip-excluded ()
  "Skip agenda items on dates in EXCLUDE property"
  (when (and (boundp 'date) date)
    (let ((excluded-string (thj/get-property "EXCLUDE"))
          exclusions)
      (when excluded-string
        (setq exclusions (mapcar
                          (lambda (ex-date-string)
                            (equal date (org-date-to-gregorian
                                          (org-parse-time-string ex-date-string))))
                          (split-string excluded-string ",")))
        (when (memq t exclusions)
          (org-end-of-subtree t))))))

(Should probably be updated to use the org native property extraction
mechanism, but this predates my discovery of that).

> On the other hand, I'm not sure about the END_DATE property. Couldn't
> DEADLINE be used for that matter? For example, `repeater-end-date'
> could be added to `org-icalendar-use-deadline' possible values. When
> this symbol belongs to variable's value and current entry has a
> deadline, any timestamps with a repeater get deadline's value as its
> END DATE property.
>
> What do you think?

Sure, that would work. Two possible issues spring to mind:

1. Is there any use cases where someone might want a deadline (in its
current use) along with an end date? Or some other conflict?

2. Will using the DEADLINE field make it harder to add exclusion from
the agenda? (I've never really used the deadline field, so don't know
what people normally use it for).

>> +(defun org-icalendar-zero-convert-timestamp (timestamp &optional format)
>> +  "Parse and format an org-formatted timestamp, zeroing the time component."
>> +  (let ((time (org-parse-time-string timestamp))
>> +	(fmt (or format "%Y%m%dT%H%M%SZ")))
>> +    (format-time-string fmt (encode-time 0 0 0 (nth 3 time) (nth 4 time) (nth 5 time)))))
>
> Can't `org-icalendar-convert-timestamp' be used instead?

It could, conceivably. However, I couldn't find any functions that would
parse a property string into the timestamp format expected by
org-icalendar-convert-timestamp, and since I would have to modify that
function quite a bit to make it do what I needed (print a UTC timestamp
string without taking time zones into account; only the date part is
used), I thought it would be easier to just create a new function.

The above interpretation of how UTC-strings are used in the exclude
parameter may be wrong, btw. Or rather, what I've done with END_DATE
here is interpret it purely as a date stamp and ignore the time part.
Looking at the iCal RFC (I realise now), that's probably not entirely
the right interpretation. However, having the "until" value include a
time might lead to unexpected results when END_DATE is set to something
close to a repetition value.

Calling it END_TIME (or, as above DEADLINE) might get this point across
and allow for an interpretation that includes the time part. Which would
mean it would make more sense to use the org-icalendar-convert-timestamp
at least for the end date. So, how do I parse that from a string? Or
perhaps that won't be an issue if using the DEADLINE field since that is
already parsed higher up in the chain?


*ahem*, sorry if the last part became a bit scatter-brained. :)

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: Adding support for limits and exclusions to iCal recurring events

[Nicolas Goaziou]

Hello,

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Handling it in the agenda can be done by an exclusion function. I
> currently have this in my org-agenda-skip-function-global (along with
> another function to skip items after their END_DATE):
>
> (defun thj/skip-excluded ()
>   "Skip agenda items on dates in EXCLUDE property"
>   (when (and (boundp 'date) date)
>     (let ((excluded-string (thj/get-property "EXCLUDE"))
>           exclusions)
>       (when excluded-string
>         (setq exclusions (mapcar
>                           (lambda (ex-date-string)
>                             (equal date (org-date-to-gregorian
>                                           (org-parse-time-string ex-date-string))))
>                           (split-string excluded-string ",")))
>         (when (memq t exclusions)
>           (org-end-of-subtree t))))))
>
> (Should probably be updated to use the org native property extraction
> mechanism, but this predates my discovery of that).

This is a separate issue anyway: it belongs to a new thread. IMO, it's
an interesting feature to be able to stop a repeated event.

>> On the other hand, I'm not sure about the END_DATE property. Couldn't
>> DEADLINE be used for that matter? For example, `repeater-end-date'
>> could be added to `org-icalendar-use-deadline' possible values. When
>> this symbol belongs to variable's value and current entry has a
>> deadline, any timestamps with a repeater get deadline's value as its
>> END DATE property.
>>
>> What do you think?
>
> Sure, that would work. Two possible issues spring to mind:
>
> 1. Is there any use cases where someone might want a deadline (in its
> current use) along with an end date? Or some other conflict?

I would be great to have feedback from other iCalendar users about it.

> 2. Will using the DEADLINE field make it harder to add exclusion from
> the agenda? (I've never really used the deadline field, so don't know
> what people normally use it for).

I'm clearly no Agenda specialist. I thought a DEADLINE would end
a repeated event already. If it's not the case, does it make sense to
implement it too?

>>> +(defun org-icalendar-zero-convert-timestamp (timestamp &optional format)
>>> +  "Parse and format an org-formatted timestamp, zeroing the time component."
>>> +  (let ((time (org-parse-time-string timestamp))
>>> +	(fmt (or format "%Y%m%dT%H%M%SZ")))
>>> +    (format-time-string fmt (encode-time 0 0 0 (nth 3 time) (nth 4 time) (nth 5 time)))))
>>
>> Can't `org-icalendar-convert-timestamp' be used instead?
>
> It could, conceivably. However, I couldn't find any functions that would
> parse a property string into the timestamp format expected by
> org-icalendar-convert-timestamp,

I agree it's not obvious, but:

  (car (org-element-parse-secondary-string "<2013-07-24 Wed>" '(timestamp)))

should do the job.

You can set minutes and hours to 0 in the returned object, but it's not
easier than creating a new function indeed.


Regards,

-- 
Nicolas Goaziou

Re: Adding support for limits and exclusions to iCal recurring events

[Michael Brand]

Hi Nicolas

On Wed, Jul 24, 2013 at 10:38 AM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> I'm clearly no Agenda specialist. I thought a DEADLINE would end
> a repeated event already.

No, DONE shifts it to the repeat date and like this it will never end.

> If it's not the case, does it make sense to implement it too?

In this case I would vote for to use a new property.

An example like

* TODO send weekly report
  DEADLINE: <2013-07-27 Sat 18:00 ++1w -1d>

could then be terminated with

* TODO send weekly report until end of terminated contract
  DEADLINE: <2013-07-27 Sat 18:00 ++1w -1d>
  END_DATE: <2013-12-31 Tue>

When implementing this, consider also whether the END_DATE should be an
agenda entry on its own and of which kind, warning period etc. I tried
to make an example that shows this issue.

Michael

Re: Adding support for limits and exclusions to iCal recurring events

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]

Michael Brand <michael.ch.brand@gmail.com> writes:

> When implementing this, consider also whether the END_DATE should be
> an agenda entry on its own and of which kind, warning period etc. I
> tried to make an example that shows this issue.

Adding to this, as mentioned previously, I interpret the iCal standard
to really permit and end *time* rather than an end *date*. Which would
make more sense in an org context? Going for an END_TIME parameter, and
then comparing exactly to the scheduled time (i.e. if current iteration
of the recurring entry > END_TIME, then filter it), or doing an END_DATE
and then interpreting the actual cut-off to be at 00:00:00 on that date?

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: Adding support for limits and exclusions to iCal recurring events

[Nicolas Goaziou]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Michael Brand <michael.ch.brand@gmail.com> writes:
>
>> When implementing this, consider also whether the END_DATE should be
>> an agenda entry on its own and of which kind, warning period etc. I
>> tried to make an example that shows this issue.
>
> Adding to this, as mentioned previously, I interpret the iCal standard
> to really permit and end *time* rather than an end *date*. Which would
> make more sense in an org context? Going for an END_TIME parameter, and
> then comparing exactly to the scheduled time (i.e. if current iteration
> of the recurring entry > END_TIME, then filter it), or doing an END_DATE
> and then interpreting the actual cut-off to be at 00:00:00 on that
> date?

I think it would be less ambiguous to use ICALENDAR_UNTIL (or UNTIL),
and apply RFC 5545:

  The value of the UNTIL rule part MUST have the same value type as the
  "DTSTART" property. Furthermore, if the "DTSTART" property is
  specified as a date with local time, then the UNTIL rule part MUST
  also be specified as a date with local time. If the "DTSTART" property
  is specified as a date with UTC time or a date with local time and
  time zone reference, then the UNTIL rule part MUST be specified as
  a date with UTC time.


Regards,

-- 
Nicolas Goaziou

Re: Adding support for limits and exclusions to iCal recurring events

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I think it would be less ambiguous to use ICALENDAR_UNTIL (or UNTIL),
> and apply RFC 5545:

Ah, that specifies it nicely, thanks (and that was what I was (trying
to) do with the patch). That would mean exporting the time as well as
the date for end times (and calling the property END_TIME, I suppose).
I would expect most people to just set and end date (i.e. a datestamp
without a time set); but that would also be supported by just going with
an end date.

I'll post an updated version of the patch for the iCal side of this
support once I get the time. Supporting it in the agenda will have to be
a separate patch since I'll probably need to do a bit more digging to
figure out how to do that properly; pointers? :)

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: Adding support for limits and exclusions to iCal recurring events

[David Rogers]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I'm clearly no Agenda specialist. I thought a DEADLINE would end
> a repeated event already. If it's not the case, does it make sense to
> implement it too?

I consider DEADLINE, in our context, to mean "when my finished work will
be needed", and therefore "when to start bothering me about getting it
done". I think that was the intention when this system was set up; in
other words, I think DEADLINE is treated as simply a personal warning
system, and it doesn't have the power to modify the agenda like TODO vs
DONE.

-- 
David R