Leap-year bug with todo-cycle

Leap-year bug with todo-cycle

[Anton Haglund]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

Hi!

I think I have discovered a possible leap-year bug in todo-cycle.

I have a TODO-entry which looks like this:

SCHEDULED: <2024-02-29 Thu ++1y>

When I cycle the TODO-entry with c-c c-t it becomes

SCHEDULED: <2025-03-01 Sat ++1y>

In my opinion it should become "2025-02-28 Fri" instead. If you don't
agree please give me the reasoning behind this.

Best regards,
Anton Haglund


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?) (was: Leap-year bug with todo-cycle)

[Ihor Radchenko]

Anton Haglund <aoh@lysator.liu.se> writes:

> I think I have discovered a possible leap-year bug in todo-cycle.
>
> I have a TODO-entry which looks like this:
>
> SCHEDULED: <2024-02-29 Thu ++1y>
>
> When I cycle the TODO-entry with c-c c-t it becomes
>
> SCHEDULED: <2025-03-01 Sat ++1y>

This is expected. When we try to add 1 year to 2024-02-29, it is
2025-02-29. However, because 02-29 does not exist in 2025, we glibc
takes the closest existing date and adds the difference in days:
2025-02-28 + 1d = 2025-03-01.

We apply the same logic to +1m repeaters:

SCHEDULED: <2024-05-31 Fri ++1m>
will become
SCHEDULED: <2024-07-01 Mon ++1m>
since 2024-06-31 does not exist.

> In my opinion it should become "2025-02-28 Fri" instead.

Keeping "end of a month" being end of a month is indeed an alternative
approach in such a situation. Both are possible; we just use the one
that is easier to implement from technical perspective.

Generally, I did see several requests to change the strategy when
calculating next month/year. However, that would be a breaking change.
I'd only go for it if people are strongly in favor of the change.
So, changing this to a poll.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?) (was: Leap-year bug with todo-cycle)

[Russell Adams]

On Fri, Apr 05, 2024 at 06:34:29PM +0000, Ihor Radchenko wrote:
> > In my opinion it should become "2025-02-28 Fri" instead.
>
> Keeping "end of a month" being end of a month is indeed an alternative
> approach in such a situation. Both are possible; we just use the one
> that is easier to implement from technical perspective.

So the poll is asking:

 If leap date doesn't exist, round date down instead of up?

I think that makes sense, at least it stays the same month.

+1

Thanks.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com
                                    https://www.adamsinfoserv.com/

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?) (was: Leap-year bug with todo-cycle)

[jman]

> Generally, I did see several requests to change the strategy when
> calculating next month/year. However, that would be a breaking change.
> I'd only go for it if people are strongly in favor of the change.
> So, changing this to a poll.

I don't particularly have a skin in the game but I ask a question: what would be the impact of this breaking change for users with existing Orgmode documents?

thanks

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?) (was: Leap-year bug with todo-cycle)

[Ihor Radchenko]

jman <emacs-orgmode@city17.xyz> writes:

> I don't particularly have a skin in the game but I ask a question: what would be the impact of this breaking change for users with existing Orgmode documents?

Mostly that people familiar with the current behaviour might be
surprised.

I am personally slightly in favour of the change, and I do not see
obvious downsides other than a surprise, but Org mode is used by many
people with various, sometimes surprising, workflows...

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Max Nikulin]

On 06/04/2024 01:34, Ihor Radchenko wrote:
> Generally, I did see several requests to change the strategy when
> calculating next month/year. However, that would be a breaking change.
> I'd only go for it if people are strongly in favor of the change.
> So, changing this to a poll.

I think the following should be taken into account: behavior of popular 
calendar applications, specifications they implement, libraries that 
likely used to create such applications.

Should it be configurable per user, per file, or even per rule 
(timestamp with repeater)?

I am not familiar with RFC5545 iCalendar, so I am unsure what options it 
recommends.

An example of recently designed library (however Org does not support 
calendars with leap *months*):
<https://tc39.es/proposal-temporal/docs/calendar.html#handling-unusual-dates-leap-days-leap-months-and-skipped-or-repeated-periods>
"Handling unusual dates: leap days, leap months, and skipped or repeated 
periods" (Temporal proposal for JavaScript)

> When Temporal encounters inputs representing a month and/or day that
> doesn't exist in the desired calendar year, by default (overridable in
> with or from via the overflow option) the inputs will be adjusted using
> the following algorithm:
> 
> - First, pick the closest day in the same month. If there are two
> equally-close dates in that month, pick the later one.
> - If the month is a leap month that doesn't exist in the desired year,
> then pick another date according to the cultural conventions of that
> calendar's users. Usually this will result in the same day in the month
> before or the month after where that month would normally fall in a leap
> year.
> - Otherwise, pick the closest date to the provided date that is still in
> the same year. If there are two equally-close dates, pick the later one.
> - If the entire year doesn't exist, then pick the closest date to the
> provided date. If there are two equally-close dates, pick the later one.

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> I think the following should be taken into account: behavior of popular 
> calendar applications, specifications they implement, libraries that 
> likely used to create such applications.

### what happens in google calendar if you repeat task monthly at 31st day of month and the next month has fewer days?

ChatGPT:

If you set a task to repeat monthly on the 31st day of the month in Google Calendar, and the next month has fewer days (e.g., February or some months with 30 days), the task will fall on the last day of that month.

### 

It is the same as the change proposed in the POLL.

> Should it be configurable per user, per file, or even per rule 
> (timestamp with repeater)?

We ought to have a switch to the existing behaviour, but I do not think
that we need it per-file or per-timestamp. If we do need such
granularity, it implies that some people actually prefer the existing
behaviour - something we are yet to hear in the replies to this poll.

> An example of recently designed library (however Org does not support 
> calendars with leap *months*):
> <https://tc39.es/proposal-temporal/docs/calendar.html#handling-unusual-dates-leap-days-leap-months-and-skipped-or-repeated-periods>
> "Handling unusual dates: leap days, leap months, and skipped or repeated 
> periods" (Temporal proposal for JavaScript)

This also picks the closest existing date - the change we propose in
this POLL.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?) (was: Leap-year bug with todo-cycle)

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 1629 bytes --]

Ihor Radchenko <yantar92@posteo.net> writes:

>> I have a TODO-entry which looks like this:
>>
>> SCHEDULED: <2024-02-29 Thu ++1y>
>>
>> When I cycle the TODO-entry with c-c c-t it becomes
>>
>> SCHEDULED: <2025-03-01 Sat ++1y>
>
> This is expected. When we try to add 1 year to 2024-02-29, it is
> 2025-02-29. However, because 02-29 does not exist in 2025, we glibc
> takes the closest existing date and adds the difference in days:
> 2025-02-28 + 1d = 2025-03-01.
>
> We apply the same logic to +1m repeaters:
>
> SCHEDULED: <2024-05-31 Fri ++1m>
> will become
> SCHEDULED: <2024-07-01 Mon ++1m>
> since 2024-06-31 does not exist.
>
>> In my opinion it should become "2025-02-28 Fri" instead.

Given the positive responses on changing the date rounding, I went ahead
and tried to implement it (see the attached; note that some tests still
need to be fixed to address the below divergence in edge cases).

However, there are still some issues remaining.
When updating timestamps repeating monthly across months with 30, 31,
and 28 days we get 

<2025-01-31 Fri +1m>
<2025-02-28 Fri +1m>
<2025-03-28 Fri +1m>
...
<2026-01-28 Wed +1m>

As you can see, because we pass through February that only has 28 days,
the timestamp tends to drift towards 28th within one year.

With the existing approach the drift would not be much better though:

<2025-01-31 Fri +1m>
<2025-03-03 Mon +1m>
<2025-03-03 Mon +1m>
...
<2026-01-03 Sat +1m>

I am wondering if we should do something with this kind of edge case.
(Not that the proposed patch is going to make things worse, but maybe
you have some ideas on what can be done, while we are at it)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Make-m-y-repeater-intervals-round-down-from-non-exis.patch --]
[-- Type: text/x-patch, Size: 10204 bytes --]

From 99e4d3c0afd438499ab55314d30a01da54b15d53 Mon Sep 17 00:00:00 2001
Message-ID: <99e4d3c0afd438499ab55314d30a01da54b15d53.1715594311.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Mon, 13 May 2024 11:36:09 +0300
Subject: [PATCH] Make m/y repeater intervals round down from non-existing
 calendar dates

* lisp/org.el (org-repeat-round-time): New customization controlling
the new behavior.  It allows falling back to the historic rounding.
(org-time-inc): New helper function to increment date by Xm/d/w/m/y.
The new function, when `org-repeat-round-time' is non-nil, uses the
closest earlier existing calendar date when repeater units are month
or year.  Otherwise, it relies upon Emacs' rounding of non-existing
calendar dates being transferred to the next month's existing dates.
(org-timestamp-change): Use the new helper function.
(org-closest-date): Use the new helper function when computing
the expected closest repeater date.
* etc/ORG-NEWS (Repeater intervals in the units of month or year are
now computed as in many other calendar apps): Document the change.

Link: https://orgmode.org/list/87frvzodze.fsf@localhost
---
 etc/ORG-NEWS |  19 ++++++++
 lisp/org.el  | 127 ++++++++++++++++++++++++++++-----------------------
 2 files changed, 88 insertions(+), 58 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 87b72ad12..8f4e51734 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -13,6 +13,25 @@ Please send Org bug reports to mailto:emacs-orgmode@gnu.org.
 
 * Version 9.7 (not released yet)
 ** Important announcements and breaking changes
+*** Repeater intervals in the units of month or year are now computed as in many other calendar apps
+
+Previously, timestamps like [2024-05-31 Fri +1m], when the next month
+does not have 31st day, were repeated to the first days of the
+following month: [2024-07-01 Mon +1m].  Same for years, when the same
+month next year does not have specified date.
+
+Now, the behavior is consistent with many common calendar apps - the
+closest /existing/ earlier date is selected: [2024-05-31 Fri +1m]
+repeats to [2024-06-30 Sun +1m].
+
+The previous behavior can be restored by customizing new option -
+~org-repeat-round-time~.
+
+Do note, however, that timestamps initially pointing to the last day
+of the month will not remain on the last day of the following months:
+[2024-05-31 Fri +1m] -> [2024-06-30 Sun +1m] -> [2024-07-30 Tue +1m]
+(not the last day anymore).
+
 *** ~org-create-file-search-functions~ can use ~org-list-store-props~ to suggest link description
 
 In Org <9.0, ~org-create-file-search-functions~ could set ~description~
diff --git a/lisp/org.el b/lisp/org.el
index 598b4ca23..81ac307cf 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -14951,7 +14951,7 @@ (defun org-diary-to-ical-string (frombuf)
     rtn))
 
 (defun org-closest-date (start current prefer)
-  "Return closest date to CURRENT starting from START.
+  "Return closest absolute date to CURRENT starting from START.
 
 CURRENT and START are both time stamps.
 
@@ -14961,12 +14961,19 @@ (defun org-closest-date (start current prefer)
 
 Only time stamps with a repeater are modified.  Any other time
 stamp stay unchanged.  In any case, return value is an absolute
-day number."
+day number.
+
+The return value is the number of days elapsed since the imaginary
+Gregorian date Sunday, December 31, 1 BC, as returned by
+`time-to-days'."
   (if (not (string-match "\\+\\([0-9]+\\)\\([hdwmy]\\)" start))
       ;; No repeater.  Do not shift time stamp.
       (time-to-days (org-time-string-to-time start))
-    (let ((value (string-to-number (match-string 1 start)))
-	  (type (match-string 2 start)))
+    (let* ((value (string-to-number (match-string 1 start)))
+	   (type (match-string 2 start))
+           (type-unit (pcase type
+                        ("h" 'hour) ("d" 'day) ("w" 'week)
+                        ("m" 'month) ("y" 'year))))
       (if (= 0 value)
 	  ;; Repeater with a 0-value is considered as void.
 	  (time-to-days (org-time-string-to-time start))
@@ -14993,50 +15000,17 @@ (defun org-closest-date (start current prefer)
 	       (let ((value (if (equal type "w") (* 7 value) value)))
 		 (setf n1 (+ sday (* value (/ (- cday sday) value))))
 		 (setf n2 (+ n1 value))))
-	      ("m"
-	       (let* ((add-months
-		       (lambda (d n)
-			 ;; Add N months to gregorian date D, i.e.,
-			 ;; a list (MONTH DAY YEAR).  Return a valid
-			 ;; gregorian date.
-			 (let ((m (+ (nth 0 d) n)))
-			   (list (mod m 12)
-				 (nth 1 d)
-				 (+ (/ m 12) (nth 2 d))))))
-		      (months		; Complete months to TARGET.
-		       (* (/ (+ (* 12 (- (nth 2 target) (nth 2 base)))
-				(- (nth 0 target) (nth 0 base))
-				;; If START's day is greater than
-				;; TARGET's, remove incomplete month.
-				(if (> (nth 1 target) (nth 1 base)) 0 -1))
-			     value)
-			  value))
-		      (before (funcall add-months base months)))
-		 (setf n1 (calendar-absolute-from-gregorian before))
-		 (setf n2
-		       (calendar-absolute-from-gregorian
-			(funcall add-months before value)))))
-	      (_
-	       (let* ((d (nth 1 base))
-		      (m (nth 0 base))
-		      (y (nth 2 base))
-		      (years		; Complete years to TARGET.
-		       (* (/ (- (nth 2 target)
-				y
-				;; If START's month and day are
-				;; greater than TARGET's, remove
-				;; incomplete year.
-				(if (or (> (nth 0 target) m)
-					(and (= (nth 0 target) m)
-					     (> (nth 1 target) d)))
-				    0
-				  1))
-			     value)
-			  value))
-		      (before (list m d (+ y years))))
-		 (setf n1 (calendar-absolute-from-gregorian before))
-		 (setf n2 (calendar-absolute-from-gregorian
-			   (list m d (+ (nth 2 before) value)))))))
+	      ((or "m" "y")
+	       (let* ((running-date (org-parse-time-string start))
+                      (next-date (org-time-inc type-unit value running-date))
+                      (current-date (org-parse-time-string current)))
+                 (while (not (time-less-p (org-encode-time current-date)
+                                        (org-encode-time next-date)))
+                   (setq running-date next-date
+                         next-date (org-time-inc type-unit value running-date)))
+                 (setf n1 (time-to-days (org-encode-time running-date))
+                       n2 (time-to-days (org-encode-time next-date)))))
+	      (_ (error "Unsupported repeater type: %S" type)))
 	    ;; Handle PREFER parameter, if any.
 	    (cond
 	     ((eq prefer 'past)   (if (= cday n2) n2 n1))
@@ -15193,6 +15167,52 @@ (defun org-at-clock-log-p ()
         (save-match-data (org-element-at-point))
         'clock)))
 
+(defcustom org-repeat-round-time t
+  "When non-nil, adjust repeated date down if it points to non-existing date.
+
+For example, when the repeater is monthly, this option, when non-nil,
+makes 31 May 2024 repeat to 30 June 2024 next month, adjusting the
+date down to avoid non-existent June 31st.  When nil, the same
+repeater would instead repeat the date at July 1st, retaining the
+extra day created by adding a monthly repeater."
+  :group 'org-time
+  :type 'boolean
+  :package-version '(Org . 9.7))
+
+(defun org-time-inc (unit value time)
+  "Increment TIME by VALUE UNITs and return new decoded time.
+TIME is decoded time, as returned by `decode-time'.
+VALUE is a number.  UNIT is one of symbols `second', `minute', `hour',
+`day', `month', `year'."
+  (unless (memq unit '(second minute hour day month year))
+    (error "org-time-inc: Unknown unit %S" unit))
+  (let ((new-time
+         (decode-time
+          (org-encode-time
+           (list
+            (+ (if (eq unit 'second) value 0) (decoded-time-second time))
+            (+ (if (eq unit 'minute) value 0) (decoded-time-minute time))
+            (+ (if (eq unit 'hour) value 0)   (decoded-time-hour time))
+            (+ (if (eq unit 'day) value 0)    (decoded-time-day time))
+            (+ (if (eq unit 'month) value 0)  (decoded-time-month time))
+            (+ (if (eq unit 'year) value 0)   (decoded-time-year time))
+            (decoded-time-weekday time)
+            (if (memq unit '(day month year))
+                nil ; Avoid auto-adjustments of time when jumping across DST.
+              (decoded-time-dst time))
+            (decoded-time-zone time))))))
+    (if (not org-repeat-round-time) new-time
+      (pcase unit
+        ((or `year `month)
+         (let ((target-year (when (eq unit 'year) (+ value (decoded-time-year time))))
+               (target-month (when (eq unit 'month) (+ value (decoded-time-month time)))))
+           (when (> target-month 12) (setq target-month (mod target-month 12)))
+           (while (or (and target-year (not (equal (decoded-time-year new-time) target-year)))
+                      (and target-month (not (equal (decoded-time-month new-time) target-month))))
+             (setq new-time (org-time-inc 'day -1 new-time)))
+           new-time))
+        (_ new-time)))))
+
 (defvar org-clock-history)                     ; defined in org-clock.el
 (defvar org-clock-adjust-closest nil)          ; defined in org-clock.el
 (defun org-timestamp-change (n &optional what updown suppress-tmp-delay)
@@ -15259,16 +15279,7 @@ (defun org-timestamp-change (n &optional what updown suppress-tmp-delay)
           ;; argument is supplied - just use whatever is provided by the
           ;; prefix argument.
           (setq dm 1))
-        (setq time
-	      (org-encode-time
-               (apply #'list
-                      (or (car time0) 0)
-                      (+ (if (eq timestamp? 'minute) increment 0) (nth 1 time0))
-                      (+ (if (eq timestamp? 'hour) increment 0)   (nth 2 time0))
-                      (+ (if (eq timestamp? 'day) increment 0)    (nth 3 time0))
-                      (+ (if (eq timestamp? 'month) increment 0)  (nth 4 time0))
-                      (+ (if (eq timestamp? 'year) increment 0)   (nth 5 time0))
-                      (nthcdr 6 time0)))))
+        (setq time (org-encode-time (org-time-inc timestamp? increment time0))))
       (when (and (memq timestamp? '(hour minute))
 		 extra
 		 (string-match "-\\([012][0-9]\\):\\([0-5][0-9]\\)" extra))
-- 
2.45.0


[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Max Nikulin]

On 13/05/2024 17:07, Ihor Radchenko wrote:
> 
> However, there are still some issues remaining.
> When updating timestamps repeating monthly across months with 30, 31,
> and 28 days we get
> 
> <2025-01-31 Fri +1m>
> <2025-02-28 Fri +1m>
> <2025-03-28 Fri +1m>
> ...
> <2026-01-28 Wed +1m>

Instead of using timestamp obtained on previous step, use original 
timestamp and multiple of the interval.

I have the following in my notes. However I have not find suitable 
functions in elisp:

https://debbugs.gnu.org/54764#10
Paul Eggert. Sat, 9 Apr 2022 00:52:57 -0700
> Generally speaking, 
> when Org mode is doing calendrical calculations it should use 
> calendrical functions rather than encode-time+decode-time, which are 
> best used for time calculations not calendar calculations.

Do you have any idea what Paul was writing about?

On 13/05/2024 17:07, Ihor Radchenko wrote:
> +           (type-unit (pcase type
> +                        ("h" 'hour) ("d" 'day) ("w" 'week)
> +                        ("m" 'month) ("y" 'year))))
I do not mind against `pcase' here, I just expected something like 
`assoc' and `string-to-char'.

> +	      ((or "m" "y")
Another comment that may be ignored: `type-unit' values might be used here.

> +          (org-encode-time
> +           (list
> +            (+ (if (eq unit 'second) value 0) (decoded-time-second time))
> +            (+ (if (eq unit 'minute) value 0) (decoded-time-minute time))
> +            (+ (if (eq unit 'hour) value 0)   (decoded-time-hour time))
> +            (+ (if (eq unit 'day) value 0)    (decoded-time-day time))

Have you considered using `min' with result of `date-days-in-month' here 
(or its sibling from timezone.el)?

> +(defun org-time-inc (unit value time)
Is there a chance that support of intervals like 1h20m will be required 
later?

> +            (+ (if (eq unit 'month) value 0)  (decoded-time-month time))
> +            (+ (if (eq unit 'year) value 0)   (decoded-time-year time))
> +            (decoded-time-weekday time)
> +            (if (memq unit '(day month year))
> +                nil ; Avoid auto-adjustments of time when jumping across DST.

Sorry, but you have to use -1, otherwise

(format-time-string
  "%F %T %Z %z"
  (encode-time '(0 30 12 15 1 2000 'ignored nil "Africa/Juba"))
  "Africa/Juba")
(error "Specified time is not representable")

> +              (decoded-time-dst time))
> +            (decoded-time-zone time))))))
> +    (if (not org-repeat-round-time) new-time
I am in doubts if `org-time-inc' should access `org-repeat-round-time' 
directly or its value should be passed as an explicit argument. Perhaps 
the additional argument may be optional with fallback to 
`org-repeat-round-time' when it is omitted.

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 13/05/2024 17:07, Ihor Radchenko wrote:
>> 
>> However, there are still some issues remaining.
>> When updating timestamps repeating monthly across months with 30, 31,
>> and 28 days we get
>> 
>> <2025-01-31 Fri +1m>
>> <2025-02-28 Fri +1m>
>> <2025-03-28 Fri +1m>
>> ...
>> <2026-01-28 Wed +1m>
>
> Instead of using timestamp obtained on previous step, use original 
> timestamp and multiple of the interval.

This is not possible because of how `org-auto-repeat-maybe' is designed.
When repeater is triggered, the original date in the timestamp is
replaced with future date. So, we have no access to the original date in
the timestamp.

> I have the following in my notes. However I have not find suitable 
> functions in elisp:
>
> https://debbugs.gnu.org/54764#10
> Paul Eggert. Sat, 9 Apr 2022 00:52:57 -0700
>> Generally speaking, 
>> when Org mode is doing calendrical calculations it should use 
>> calendrical functions rather than encode-time+decode-time, which are 
>> best used for time calculations not calendar calculations.
>
> Do you have any idea what Paul was writing about?

No idea. I am not aware about any existing built-in API that provides
date increments.

>> +          (org-encode-time
>> +           (list
>> +            (+ (if (eq unit 'second) value 0) (decoded-time-second time))
>> +            (+ (if (eq unit 'minute) value 0) (decoded-time-minute time))
>> +            (+ (if (eq unit 'hour) value 0)   (decoded-time-hour time))
>> +            (+ (if (eq unit 'day) value 0)    (decoded-time-day time))
>
> Have you considered using `min' with result of `date-days-in-month' here 
> (or its sibling from timezone.el)?

Not sure if it is going to be simpler.

>> +(defun org-time-inc (unit value time)
> Is there a chance that support of intervals like 1h20m will be required 
> later?

Not sure again. I simply used ts-inc function signature from Adam's
ts.el as reference.

>> +            (+ (if (eq unit 'month) value 0)  (decoded-time-month time))
>> +            (+ (if (eq unit 'year) value 0)   (decoded-time-year time))
>> +            (decoded-time-weekday time)
>> +            (if (memq unit '(day month year))
>> +                nil ; Avoid auto-adjustments of time when jumping across DST.
>
> Sorry, but you have to use -1, otherwise
>
> (format-time-string
>   "%F %T %Z %z"
>   (encode-time '(0 30 12 15 1 2000 'ignored nil "Africa/Juba"))
>   "Africa/Juba")
> (error "Specified time is not representable")

Hmm. I am not sure if it is a real problem in practice.
String values of time zone are only possible for hand-crafted time
values. Using `decode-time' does not retain the time zone name:

(decode-time
  (encode-time '(0 30 12 15 1 2000 'ignored -1 "Africa/Juba")))
; => (0 30 12 15 1 2000 6 nil 7200)

The reason why I forced DST nil is hysteresis of glibc when dealing with
DST transitions:

(cl-loop for m in '(1 2 3 4 5 6 7 8 9 10 11 12)
  collect (decode-time (encode-time `(0 0 0 4 ,m 2012 nil -1 10800))))

(0 0 23 3 1 2012 2 nil 7200)
(0 0 23 3 2 2012 5 nil 7200)
(0 0 23 3 3 2012 6 nil 7200)
(0 0 0  4 4 2012 3 t 10800)
(0 0 0  4 5 2012 5 t 10800)
(0 0 0  4 6 2012 1 t 10800)
(0 0 0  4 7 2012 3 t 10800)
(0 0 0  4 8 2012 6 t 10800)
(0 0 0  4 9 2012 2 t 10800)
(0 0 0  4 10 2012 4 t 10800)
(0 0 23 3 11 2012 6 nil 7200)
(0 0 23 3 12 2012 1 nil 7200)

Although, forcing DST nil will not always help here, so I need some
other solution to this problem.
(timezones, DSTs... :sigh:)

>> +              (decoded-time-dst time))
>> +            (decoded-time-zone time))))))
>> +    (if (not org-repeat-round-time) new-time
> I am in doubts if `org-time-inc' should access `org-repeat-round-time' 
> directly or its value should be passed as an explicit argument. Perhaps 
> the additional argument may be optional with fallback to 
> `org-repeat-round-time' when it is omitted.

This is of no use for now - we do not need to pass different parameters
to `org-time-inc' from the existing Org code. But we can always add an
extra optional argument if necessary.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Stefan Nobis]

Ihor Radchenko <yantar92@posteo.net> writes:

> This is not possible because of how `org-auto-repeat-maybe' is
> designed. When repeater is triggered, the original date in the
> timestamp is replaced with future date. So, we have no access to the
> original date in the timestamp.

And even if we would have access to the original date: It may be
something like the last day of February and from that alone it is not
evident, wether e.g. the 28. of each month should be used or the last
day of month.

Therefore, I think, the only solution to this problem is to have some
kind of special 'last day of month' marker. Maybe something like
"+1m!" or any other way of encoding this "go to the last day of the
next month" directly into the increment specifier.

-- 
Until the next mail...,
Stefan.

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Ihor Radchenko]

Stefan Nobis <stefan-ml@snobis.de> writes:

> ...
> And even if we would have access to the original date: It may be
> something like the last day of February and from that alone it is not
> evident, wether e.g. the 28. of each month should be used or the last
> day of month.
>
> Therefore, I think, the only solution to this problem is to have some
> kind of special 'last day of month' marker. Maybe something like
> "+1m!" or any other way of encoding this "go to the last day of the
> next month" directly into the increment specifier.

+1m! is awkward - what if [2024-05-12 +1m!]?

One option might be something like [2024-05--1 +1m] that will work
nicely with glibc's time rounding conventions:

(format-time-string "%Y-%m-%d" (encode-time '(0 0 0 -1 5 2024))) ; => 2024-04-29

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Stefan Nobis]

Ihor Radchenko <yantar92@posteo.net> writes:

> +1m! is awkward - what if [2024-05-12 +1m!]?

You are right. I have not really spent time to think about a nice
syntax. But I think it needs to be encoded in the increment part, not
in the date part (because after the first increment, the information
that we want the last day of each month would be lost; and always
using yyyy-(mm+1)--1 also seems a bit awkward).

So maybe "+1mx" or some other way to encode "go to last day of next
month" directly into the increment?

-- 
Until the next mail...,
Stefan.

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Ihor Radchenko]

Stefan Nobis <stefan-ml@snobis.de> writes:
> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> +1m! is awkward - what if [2024-05-12 +1m!]?
>
> You are right. I have not really spent time to think about a nice
> syntax. But I think it needs to be encoded in the increment part, not
> in the date part (because after the first increment, the information
> that we want the last day of each month would be lost; and always
> using yyyy-(mm+1)--1 also seems a bit awkward).

It does not have to be lost. We can adjust the repeater function to
handle negative dates/months specially, so that

[2024-05--1 +1m] -> [2024-06--1 +1m]

> So maybe "+1mx" or some other way to encode "go to last day of next
> month" directly into the increment?

How does it make my example with [2024-05-12 +1m!] any less awkward?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Stefan Nobis]

Ihor Radchenko <yantar92@posteo.net> writes:

> It does not have to be lost. We can adjust the repeater function to
> handle negative dates/months specially, so that

> [2024-05--1 +1m] -> [2024-06--1 +1m]

Hmmm... I do not like, that here the wrong month is displayed. On a
cursory look, it's easy to assume the wrong month. But it is an
interesting option and maybe misreading this only my personal problem
(and "--1" stands out a little bit, so it is recognizable).

> How does it make my example with [2024-05-12 +1m!] any less awkward?

Ups, I misunderstood you. I thought the "+1m!]" looks ugly (and maybe
"+1mx]" is a bit less ugly).

Now I see what you mean. But there are new possibilities! :) Start the
repeater now, but from the next month on always use the last day of
each month. But yes, that's seems to be a quite seldom use-case.

But never mind, I have not thought this all through. Only some random
musings. :)

-- 
Until the next mail...,
Stefan.

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Ihor Radchenko]

Stefan Nobis <stefan-ml@snobis.de> writes:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> It does not have to be lost. We can adjust the repeater function to
>> handle negative dates/months specially, so that
>
>> [2024-05--1 +1m] -> [2024-06--1 +1m]
>
> Hmmm... I do not like, that here the wrong month is displayed. On a
> cursory look, it's easy to assume the wrong month. But it is an
> interesting option and maybe misreading this only my personal problem
> (and "--1" stands out a little bit, so it is recognizable).

Technically, nothing stops us from interpreting 2024-05--1 as "last day
of month 05". But the nice thing about interpreting it as previous month
is that it works "magically" with non-Org Elisp APIs like
`parse-time-string':

(parse-time-string "2024-09--1") is able to parse the date.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Max Nikulin]

On 14/05/2024 19:56, Ihor Radchenko wrote:
> Max Nikulin writes:
>> On 13/05/2024 17:07, Ihor Radchenko wrote:
>>>
>>> <2025-01-31 Fri +1m>
>>> <2025-02-28 Fri +1m>
>>> <2025-03-28 Fri +1m>
>>
>> Instead of using timestamp obtained on previous step, use original
>> timestamp and multiple of the interval.
> 
> This is not possible because of how `org-auto-repeat-maybe' is designed.

Then the only way to get reasonable results is to store in :PROPERTIES: 
either original date (and iteration count) or current date before 
clamping (2025-02-31)

> No idea. I am not aware about any existing built-in API that provides
> date increments.

The context was that day start time may give some surprise as well. I am 
unsure concerning namely increments. I just expected a set of more 
accurate functions for working with dates.

>>> +          (org-encode-time
[...]
>>> +            (+ (if (eq unit 'day) value 0)    (decoded-time-day time))
>>
>> Have you considered using `min' with result of `date-days-in-month' here
>> (or its sibling from timezone.el)?
> 
> Not sure if it is going to be simpler.

My idea was to avoid the backward `while' loop in the code below this line.

>> (format-time-string
>>    "%F %T %Z %z"
>>    (encode-time '(0 30 12 15 1 2000 'ignored nil "Africa/Juba"))
>>    "Africa/Juba")
>> (error "Specified time is not representable")
> 
> Hmm. I am not sure if it is a real problem in practice.
> String values of time zone are only possible for hand-crafted time
> values.

This example avoids changing system time or starting Emacs with TZ 
environment. The following example does not include explicit timezone:

TZ=Africa/Juba emacs -Q --batch \
     --eval "(encode-time '(0 30 12 15 1 2000 'ignored nil nil))"

> The reason why I forced DST nil is hysteresis of glibc when dealing with
> DST transitions:

At first I was trying to figure out what time zone had 24:00<->23:00 
transition in 2012.

> (cl-loop for m in '(1 2 3 4 5 6 7 8 9 10 11 12)
>    collect (decode-time (encode-time `(0 0 0 4 ,m 2012 nil -1 10800))))
> 
> (0 0 23 3 1 2012 2 nil 7200)

This result depends on your timezone. It is consistent with e.g. 
Asia/Istanbul. It had +02:00 offset in January, but you requested 
+03:00. The result of conversion is -1 hour difference in requested 
timestamp and conversion result. It is unrelated to internal DST state 
and hysteresis caused by this state.

When DST is nil
2012-01-04 00:00:00 +03:00 === 2012-01:03 23:00:00 +02:00

Avoid passing timezone offset when you do not know it.

I agree that there are issues with handling tm_gmtoff and tm_isdst
https://data.iana.org/time-zones/theory.html#POSIX-extensions
Results may vary across libraries.

> Although, forcing DST nil will not always help here

It is fragile, I would not rely on ignoring offset when DST is 
specified. Actually you pass mutually inconsistent values, so it is 
either undefined behavior or close to it.

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>>> On 13/05/2024 17:07, Ihor Radchenko wrote:
>>>>
>>>> <2025-01-31 Fri +1m>
>>>> <2025-02-28 Fri +1m>
>>>> <2025-03-28 Fri +1m>
>>>
>>> Instead of using timestamp obtained on previous step, use original
>>> timestamp and multiple of the interval.
>> 
>> This is not possible because of how `org-auto-repeat-maybe' is designed.
>
> Then the only way to get reasonable results is to store in :PROPERTIES: 
> either original date (and iteration count) or current date before 
> clamping (2025-02-31)

As Stefan pointed, even this is not a panacea - 2025-02-28 starting date
is simply ambiguous because it may either mean "last day of each month"
or "28th of each month".

Now, I am having second thoughts about how useful the customization I
proposed is - it looks like it is only changing one kind of unexpected
behavior with another.

I still want to work towards applying my patch for the purposes of
making agenda calculations for the repeater consistent with reality
(without the patch, `org-closest-date' and `org-timestamp-change' yield
different results because the former moves multiple repeater intervals
forward without considering date jumps), but I think that the
customization should be simply dropped.

>>> Have you considered using `min' with result of `date-days-in-month' here
>>> (or its sibling from timezone.el)?
>> 
>> Not sure if it is going to be simpler.
>
> My idea was to avoid the backward `while' loop in the code below this line.

That while loop is going to have 3 iterations at worst. So, I do not see
much benefit trading performance for complexity.

>> Although, forcing DST nil will not always help here
>
> It is fragile, I would not rely on ignoring offset when DST is 
> specified. Actually you pass mutually inconsistent values, so it is 
> either undefined behavior or close to it.

Yeah. I think that the easiest approach will be converting to UTC first,
making the shift, and then converting back to the original
timezone. That way, we will avoid issues with moving across DST
transition.

Also, special handling may be necessary for timestamps without time -
currently they are simply treated as 0:00 time, but it may backfire as
in my case with DST transition moving the date one day back.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Max Nikulin]

On 14/05/2024 19:56, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>>> +(defun org-time-inc (unit value time)
>> Is there a chance that support of intervals like 1h20m will be required
>> later?
> 
> Not sure again. I simply used ts-inc function signature from Adam's
> ts.el as reference.

ts.el has `ts-adjust' that may change several fields. I had in mind 
specification of time interval from RFC5545 iCalendar P4h20m.

>>> +            (if (memq unit '(day month year))
>>> +                nil ; Avoid auto-adjustments of time when jumping across DST.
>>
>> Sorry, but you have to use -1, otherwise
>>
>> (format-time-string
>>    "%F %T %Z %z"
>>    (encode-time '(0 30 12 15 1 2000 'ignored nil "Africa/Juba"))
>>    "Africa/Juba")
>> (error "Specified time is not representable")
> 
> Hmm. I am not sure if it is a real problem in practice.

You are right. When you are using TZ as integer offset from UTC, 
`encode-time' does not signal any error. It ignores DST. However it is 
the case of conversion to local time a timestamp with arbitrary explicit 
offset, e.g. taken from a Date email header. So my example is wrong for 
current patch revision, but the code needs a fix.

You need to get local timestamp for given local time. You should use -1 
for DST and nil for TZ.

In some cases it may cause change like 1:30 to 2:30, but it is not worse 
than date drift from 31 to 30 or even 28.

> The reason why I forced DST nil is hysteresis of glibc when dealing with
> DST transitions:

As I wrote yesterday, you example is unrelated to glibc hysteresis. The 
reliable way to deal with ambiguous time close to time transition is to 
have get next transition function for given timezone and a way to 
resolve ambiguous time (DST can not help in some cases). Heuristics with 
trying a day before and a day after should work, but it may impact 
performance.

Your patch allows users to choose if they prefer May, 31 to July, 1 or 
to July, 30 drift. It may be an improvement some of them.

I think, they actually expect choice to get June, 30 and July, 31 or to 
ignore June since it has 30 days only.

Re: [POLL] Dealing with +1m/y repeaters when jumping to impossible date (should 05-31 +1m be 07-01 or 06-30?)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>>>> +(defun org-time-inc (unit value time)
>>> Is there a chance that support of intervals like 1h20m will be required
>>> later?
>> 
>> Not sure again. I simply used ts-inc function signature from Adam's
>> ts.el as reference.
>
> ts.el has `ts-adjust' that may change several fields. I had in mind 
> specification of time interval from RFC5545 iCalendar P4h20m.

We do not follow RFC5545 anywhere.
The closest is org-duration, which would use 4h20min.
But then one can simply use
(org-time-inc 'minute (org-duration-to-minutes "4h20min") time)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>