bug#67331: 30.0.50; New Todo mode feature: changing item date style

[Eli Zaretskii <eliz@gnu.org>]

> From: Stephen Berman <stephen.berman@gmx.net>
> Date: Tue, 21 Nov 2023 16:32:34 +0100
> 
> This post is a followup to bug#66395, where I posted (and subsequently
> installed) a patch to support the ISO date style in Todo mode item
> headers.  As I pointed out in that bug thread, despite supporting
> different date styles, Todo mode does not support interactively and
> automatically changing between different date styles.  I have attempted
> to add this functionality to Todo mode, and the attached patch contains
> my work-in-progress implementation.  I'm still testing these changes, so
> even if they are deemed acceptable I'm not yet ready to install them,
> but they raise some questions and how I proceed will be influenced and
> perhaps decided by the answers to these questions.
> 
> The rest of this post is directed mainly to the Emacs maintainers and I
> apologize to them for its length, but unless they want to leave it
> entirely up to me how to proceed with these changes, I think they need
> to know in at least some detail what the issues are.

Let me take a step or two back, and ask whether Todo mode needs to
support different styles, let alone interactive changes between
styles?  IOW, what bad things will happen if Todo mode supported only
one fixed style?  I'm asking this because it sounds like adding this
support raises quite a few issues whose resolution is hairy at best,
and in some cases causes real user-visible problems.  Refusing to
support more than one style will eliminate all these issues, with
little or no effort.

If you do have to support the various styles, my next question is: why
not make the style a display-only feature?  That is, keep using a
single style in the TODO files, but reformat the date to a
user-specified style at display time.  This should at least avoid the
complications of incompatible files, switching between file formats,
whether and how should all the existing files be converted to the
selected style, etc.  After all, why should users care about the
internal format of how time stamps are recorded on a TODO file, if
they use Todo mode to display and edit those files?

Failing these solutions that avoid the complications, here are my
opinions about the issues you bumped into:

> - Although the use and effect of this feature is confined to
>   todo-mode.el, it depends on a making a change to calendar.el, as you
>   can see in the above summary and in the patch.  The reason, as I
>   already noted in bug#66395, is that Todo mode date styles are
>   specified by `calendar-date-display-form', which you can change by
>   customizing `calendar-date-style' or by executing
>   `calendar-set-date-style'.  To keep this connection I added a hook to
>   calendar.el callable from the latter function, that can be used to
>   trigger the date header changes in Todo mode (but it's not a hook in
>   the usual sense of being intended for user customization; rather, it's
>   intended for use by packages).  There is precedence for this in commit
>   a8f4bb8361, where I similarly added a hook to diary-lib.el for the
>   benefit of todo-mode.el.  In addition, since the hook has the value
>   nil by default, there is no change to the default behavior of the
>   Emacs Calendar (or Diary).  So, is this change to calendar.el
>   acceptable?

I think this is acceptable, but we need commentary for this hook to
explain the subtleties you mention above.

> - My implementation of this feature crucially involves a change in the
>   internal form of todo files.  Till now the first line of each todo
>   file has contained a sexp listing the categories in the file (this
>   line is hidden in todo-mode), but to allow switching date styles I
>   have augmented this sexp to also contain a specification of the
>   current date style used by the item headers in the file.  A number of
>   existing todo-mode functions make use of this sexp, so they have also
>   had to be adapted.  All these changes are strictly internal, so the
>   Todo mode UI remains unchanged.  The implementation also includes
>   automatically converting existing todo files to the augmented metadata
>   format, so that users of Todo mode can continue using existing todo
>   files, as well as the new functionality, without needing to be aware
>   of the internal changes.  But once converted, the todo files will not
>   be compatible with previous versions of todo-mode.el.  This would be a
>   problem if someone wanted to use the same todo files on different
>   systems, not all of which have an Emacs where todo-mode.el has these
>   changes.
> 
>   Is this problem considered unacceptable?

It is certainly undesirable.  But we already did something similar,
for example in the format of desktop files.  See commits c96983efef
and 20defc5538.

>   So is it acceptable to change the metadata format and the item header
>   for each todo file on demand instead of for all todo files at once?

I think so, yes.  But again, such changes are undesirable in the first
place.

> - Another thorny issue is how to deal with unit tests.  Currently, there
>   are a number of tests for todo-mode.el using ERT.  Since they all
>   require a todo file as input data and many make use of the file
>   metadata (currently in the old format), at least these tests will
>   require adjusting.  So should there be parallel tests for todo-mode
>   with the new and with the previous metadata format?  Or should the
>   older format be declared obsolete?  (However, AFAIK there is no formal
>   obsoletion mechanism for data formats as opposed to functions and
>   variables.)  I think the decision about this is contingent on the
>   decision about allowing reverting the format for backwards
>   compatibility.

If we keep supporting both formats, even if the old one is deprecated,
we will need to keep the tests for that old format.  Deprecation is
not removal, so removing the tests (which already exist) sounds like
not the best idea to me.

> Finally, I want to point out an issue that I think is mainly orthogonal
> to the new feature and the preceding considerations, but it helps
> clarify the extent of the feature.  Currently, Todo mode supports item
> date headers with either the American, European or ISO date style, but
> all items can only be displayed with the same style, and that
> restriction will persist if the changes discussed here are installed:
> then you can switch date styles, but doing so changes all item headers
> in a visited todo file.  In other words, there can be no mixing of date
> styles within a todo file, or when switching between todo files.  This
> restriction is justified by avoiding ambiguity if dates are displayed
> simultaneously in the European and American styles with numbers for
> months.  Admittedly, `calendar-date-display-form' uses month names by
> default in these styles, which avoids ambiguity (and there's no
> ambiguity between the ISO style and the other styles, unless years are
> displayed with just the two least significant digits, which is common in
> the American and European styles but AFAIK the ISO standard requires
> four digits for years).  Hence, at least for such display forms, mixing
> date display styles should be possible (though whether it's desirable is
> questionable).  I haven't given serious thought to how to implement
> that, nor to how it would interact with the proposed feature of
> switching date styles without mixing, but it would likely require
> changes to the implementation in the attached patch.

I think it's entirely reasonable to support only one style for
displaying TODO items.  I cannot think about a reason for displaying
different items using different styles.

Thanks.