[POLL] Proposed syntax for timestamps with time zone info (was: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda)

[Ihor Radchenko <yantar92@posteo.net>]

Greg Minshall <minshall@umich.edu> writes:

> just a thought/reminder.  there are "semantics" and "encoding".  a spec
> like ISO-8601 specifies both.  the important thing for org-mode is to
> use an encoding that
>
> 1. is easily parsable/understandable by the mere mortal
>
> 2. allows expression of all the semantics of the underlying spec/specs
>    (be that ISO-8601, this new IETF spec, the Library of Congress spec,
>    etc.)
>
> 3. and, importantly, is designed to *try* to follow updates to the
>    underlying spec/specs (which will inevitably happen)

I agree with these three points.

Following the previous discussion and the various links provided, I have
reviewed the main discussed timestamp standards and would like to
propose the new Org timestamp syntax that will allow specifying time
zone information.

I will not follow the standards fully because the available standards
are not designed to be easily understood by humans. I will also omit
the ideas from the standards that are unrelated to time stamps (but
still outline them below, and keep them in mind for
forward-compatibility). I will, however, try to use the syntax close to
the standards where possible.

1. https://datatracker.ietf.org/doc/draft-ietf-sedate-datetime-extended/
   proposal is extending ISO8601/RFC3339 with time zone information. In
   addition to UTC offset defined in ISO8601, it allows specifying the
   time zone identifier and auxiliary information.

   Examples:

   2022-07-08T02:14:07+02:00[Europe/Paris]
   (both offset, and time zone are specified)
   Note that we cannot use "[]" symbols because they are incompatible
   with current timestamp syntax that must not contain closing "]>"
   inside the timestamp.

   1996-12-19T16:39:57-08:00[America/Los_Angeles][u-ca=hebrew]
   (preferred calendar is specified in addition to time zone)
   Note: calendar spec is out of scope of time zone discussion - if we
   decide to add it in future, we can simply add new parts to
   timestamps, just like repeater interval and warning period.

   Further, the draft proposes an idea, which have also been discussed
   in this thread: making use of redundant UTC offset + time zone
   information to detect possible unexpected changes in time zone rules:

   2022-07-08T00:14:07+00:00[!Europe/London]
   ("!" identifies that +00:00 offset, if not consistent with
   Europe/London at the timestamp time, must be signalled to user or
   generally not ignored)

2. https://www.loc.gov/standards/datetime/ does not contain any new
   ideas relevant to time zones, although it has many other ideas wrt
   approximate/incomplete timestamps. I will outline them later, but
   they do not directly affect the proposed new Org timestamp syntax.
   
|-----------------------------------|
| The proposed new timestamp syntax |
|-----------------------------------|

I propose two forms of time zone information in Org timestamps

1. Timestamp with explicit UTC offset

   YYYY-MM-DD [optional day name] HH:MM[^ \]>]*?[+-−]HH[MM]?
   YYYY-MM-DD [optional day name] HH:MM[^ \]>]*?Z[ \]>]
   
   "-" is latin "HYPHEN-MINUS" (code 0x2D)
   "−" is unicode "MINUS SIGN" (code 0x2212), as prescribed by ISO8601
   we will not actually use it when generating timestamps, but allow it
   to keep some compatibility with ISO standard.

   "Z" in the second format refers to "Zulu" time (another name for UTC)
   and must be either the last character in the timestamp or the
   last character before space. Not sure if many users are familiar with
   "Z" convention, but it is (1) in ISO; (2) succinct for users who are
   familiar with it. We will prefer +00 number offset in auto-generated
   timestamps.

   Examples:
   2022-11-12 12:00+02 # 12:00 UTC+2
   2022-11-12 14:00+0230 # 14:00 UTC+2:30
   2022-11-12 14:00-0230 # 14:00 UTC-2:30
   2022-11-12 14:00Z # 14:00 UTC

   The offset is a subset of what is defined by ISO8601.
   
   Note that unlike ISO8601, ":" is not allowed in the offset specifier.
   This is to disambiguate with the time intervals in Org timestamps:
   [2022-11-12 8:00-9:00] in Org is a time range from 8am to 9am.

   For time ranges, we will only allow a single offset and time zone
   specifier for both start and end times:
   [2022-11-12 8:00-9:00+08]
   If different time zones are necessary to specify the start/end times,
   users can still use full timestamp range syntax
   [2022-11-12 8:00+03]--[2022-11-12 22:00+08]

   The format is also forward-compatible with extending Org timestamps
   for second/sub-second precision: 2022-11-12 14:00:05.0012+0230 will
   remain valid if we decide to adopt such format.

2. Timestamp with time zone specifier

   YYYY-MM-DD [optional day name] HH:MM[^ ]* @[!]?<[^ \]>]>

   For now, time zone name will only be processed when it follows TZ
   POSIX format. If necessary, the proposed syntax will still allow
   extensions to TZ POSIX.

   Examples:
   2022-11-12 12:00 @Asia/Singapore # tzdb syntax
   2022-11-12 10:00 @Europe/Berlin
   2022-11-12 10:00 @!Europe/Berlin # "!" does nothing here, see below
   2022-11-12 10:00 @EST+5 # TZ syntax
   2022-11-12 10:00 @EST+5EDT,M3.2.0/2,M11.1.0/2 # manual time zone spec allowed in TZ

   The "@" symbol is selected to disambiguate time zone specifier from
   other auxiliary information in the timestamp. Like calendar name,
   which might be added in future. Note that we cannot use [...] from
   the standard draft. I selected "@" because it is read as "at" -
   location specifier.

   The "!" symbol is adapted from
   https://datatracker.ietf.org/doc/draft-ietf-sedate-datetime-extended/

   I use space before "@" to improve readability. We deviate from the
   standard here so may as well. In contrast, no space before [+-]offset
   is closer to the standard yet not cluttering the timestamp too much
   (IMHO).

3. (1) and (2) can be combined

   2022-11-12 12:00+08 @Asia/Singapore

   Org will unconditionally use +08 offset and ignore the time zone
   name. We prefer absolute offset because it is non-ambiguous wrt
   out-of-date tzdb on the computer. One may also not update the TZ
   variable upon travelling - the system clock will again be more likely
   accurate.

   This redundant time offset info can serve as human-readable
   indication of absolute time, while the time zone name will indicate
   the location.

   2022-11-12 12:00+07 @!Asia/Singapore

   Org will calculate the internal time for both +07 offset and
   Asia/Singapore time zone. If they do not match, Org will issue a
   warning. The offset will still be preferred if we have to proceed.

   This can be useful when planning future meetings and sending Org file
   with a timestamp to someone else, potentially with obsolete tzdb.

|-----------------------------------|
|              <end>                |
|-----------------------------------|

Apart from the ideas mentioned above,
https://www.loc.gov/standards/datetime/ contains a number of other
interesting ideas that may or may not be adapted by Org in future.
I will outline the ideas I find noteworthy to keep them in mind when
considering changing (including current changes) in the timestamp
syntax:

1. Reduced timestamp precision:
   1985-04-12 (day precision, time omitted; available in Org)
   1985-04 (month precision, day and time omitted, not available in Org)
   1985 (year precision)

   Current timestamp syntax proposal should not interfere.

2. Using "T" as date/time delimiter
   1985-04-12T23:20:30 (not supported by Org)

3. Using "/" for time intervals
   2004-02-01/2005-02-08 (not supported and incompatible with Org)

4. Seasons
   2001-21 (Spring, 2001; not supported by Org)

5. Approximate dates
   1984? (year uncertain)
   2004-06~ (year-month approximate)
   2004-06-11% (entire date (year-month-day) uncertain and approximate)
   2004-06?-11 (year and month uncertain)
   2004-?06-11 (just month uncertain)

6. Unspecified digits
   1985-04-XX (day unspecified; might be an interesting idea for
   defining repeater intervals)

7. Open time intervals
   1985-04-12/.. (from 1985-04-12 to infinite)
   1985-04-12/ (from 1985-04-12 to unknown)

8. Negative calendar year
   -1985 (note: an alternative could be allowing AD/BC)

9. Set of dates
   [1667,1668,1670..1672] (one of)
   {1667,1668,1670..1672} (all)
   [1760-01,1760-02,1760-12..]
   (similar to regexp [...] groups; not compatible with Org syntax, but
   the idea could be interesting for repeater intervals)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>