[BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ignacio Casso]

Hello,

In Emacs 27.2, with an up to date version of org from ELPA (9.5.2),
org-agenda considers timestamps that appear in property drawers, so the
entry below appears in the daily agenda view.

* Heading
  :PROPERTIES:
  :timestamp: <2022-03-12 sáb>
  :END:

However, in the latest Emacs version built from source (29.0.50), with
the built-in version of org (also 9.5.2, but the latest release, I
assume), this is no longer the case and that entry does not appear in
the agenda view.

I know that maybe it's unorthodox, but I have some org files that rely
in the previous behavior, with entries like the following:

* Some friend
  :PROPERTIES:
  :birth-date: <1994-03-12 sáb +1y>
  :END:

Is this a bug? If it's not, can someone point me to the functions I need
to touch to restore the previous behavior? Or maybe I should stop doing
this and start moving those timestamps out of the properties drawer in
my files?

Regards,

Ignacio


Emacs  : GNU Emacs 29.0.50 (build 9, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0)
 of 2022-03-11
Package: Org mode version 9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ihor Radchenko]

Ignacio Casso <ignaciocasso@hotmail.com> writes:

> In Emacs 27.2, with an up to date version of org from ELPA (9.5.2),
> org-agenda considers timestamps that appear in property drawers, so the
> entry below appears in the daily agenda view.
>
> * Heading
>   :PROPERTIES:
>   :timestamp: <2022-03-12 sáb>
>   :END:
>
> However, in the latest Emacs version built from source (29.0.50), with
> the built-in version of org (also 9.5.2, but the latest release, I
> assume), this is no longer the case and that entry does not appear in
> the agenda view.
>
> I know that maybe it's unorthodox, but I have some org files that rely
> in the previous behavior, with entries like the following:
>
> * Some friend
>   :PROPERTIES:
>   :birth-date: <1994-03-12 sáb +1y>
>   :END:
>
> Is this a bug? If it's not, can someone point me to the functions I need
> to touch to restore the previous behavior? Or maybe I should stop doing
> this and start moving those timestamps out of the properties drawer in
> my files?

What you see in the new Org version is not a bug. Property values are
treated as plain text by Org.

In the older versions, agenda code did not rely on Org's internal
parsing and matched timestamps in places where timestamps are not
allowed (inside code blocks, for example). See
https://orgmode.org/list/20220101122409.GA29829@itccanarias.org

Dear all,

I was unable to find a place in manual describing that timestamps cannot
be placed inside property values:

>> A timestamp is a specification of a date (possibly with a time or a
>> range of times) in a special format, either ‘<2003-09-16 Tue>’ or
>> ‘<2003-09-16 Tue 09:39>’ or ‘<2003-09-16 Tue 12:00-12:30>’(1).  A
>> timestamp can appear anywhere in the headline or body of an Org tree
>> entry.  Its presence causes entries to be shown on specific dates in the
>> agenda (see *note Weekly/daily agenda::).  We distinguish:

I personally see allowing timestamps (and links) inside property values
as a useful feature. Would it be of interest for other users?

In any case, we should probably clarify manual in this regard.

Best,
Ihor

RE: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Tom Davey]

Ihor writes: 

> I personally see allowing timestamps (and links) inside property values as a useful feature. 
> Would it be of interest for other users?

Yes, it's a quite useful feature. For years, via my Capture templates, I've been adding a property named :Created: to the properties drawer as follows: 

:PROPERTIES:
:Created:  <2022-03-06 Sun 22:42>
:END:

Now, in 9.5.2, literally hundreds of entries that formerly appeared on the built-in Agenda views cannot be easily found. 

Regards to all, 
Tom 

PS The variable 'org-agenda-skip-additional-timestamps-same-entry seemed expressly made for my use case, to clean up same-day clutter in entries with a TODO timestamp.

--
Tom Davey
tom@tomdavey.com
New York NY USA

-----Original Message-----
From: Emacs-orgmode <emacs-orgmode-bounces+tom=tomdavey.com@gnu.org> On Behalf Of Ihor Radchenko
Sent: Saturday, March 12, 2022 7:29 AM
To: Ignacio Casso <ignaciocasso@hotmail.com>
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

Ignacio Casso <ignaciocasso@hotmail.com> writes:

> In Emacs 27.2, with an up to date version of org from ELPA (9.5.2), 
> org-agenda considers timestamps that appear in property drawers, so 
> the entry below appears in the daily agenda view.
>
> * Heading
>   :PROPERTIES:
>   :timestamp: <2022-03-12 sáb>
>   :END:
>
> However, in the latest Emacs version built from source (29.0.50), with 
> the built-in version of org (also 9.5.2, but the latest release, I 
> assume), this is no longer the case and that entry does not appear in 
> the agenda view.
>
> I know that maybe it's unorthodox, but I have some org files that rely 
> in the previous behavior, with entries like the following:
>
> * Some friend
>   :PROPERTIES:
>   :birth-date: <1994-03-12 sáb +1y>
>   :END:
>
> Is this a bug? If it's not, can someone point me to the functions I 
> need to touch to restore the previous behavior? Or maybe I should stop 
> doing this and start moving those timestamps out of the properties 
> drawer in my files?

What you see in the new Org version is not a bug. Property values are treated as plain text by Org.

In the older versions, agenda code did not rely on Org's internal parsing and matched timestamps in places where timestamps are not allowed (inside code blocks, for example). See https://orgmode.org/list/20220101122409.GA29829@itccanarias.org

Dear all,

I was unable to find a place in manual describing that timestamps cannot be placed inside property values:

>> A timestamp is a specification of a date (possibly with a time or a 
>> range of times) in a special format, either ‘<2003-09-16 Tue>’ or
>> ‘<2003-09-16 Tue 09:39>’ or ‘<2003-09-16 Tue 12:00-12:30>’(1).  A 
>> timestamp can appear anywhere in the headline or body of an Org tree 
>> entry.  Its presence causes entries to be shown on specific dates in 
>> the agenda (see *note Weekly/daily agenda::).  We distinguish:

I personally see allowing timestamps (and links) inside property values as a useful feature. Would it be of interest for other users?

In any case, we should probably clarify manual in this regard.

Best,
Ihor

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ignacio Casso]

[-- Attachment #1: Type: text/plain, Size: 1494 bytes --]


>> What you see in the new Org version is not a bug. Property values are
>> treated as plain text by Org.
>>
>> I was unable to find a place in manual describing that timestamps cannot
>> be placed inside property values:

>> I personally see allowing timestamps (and links) inside property values as a useful feature.
>> Would it be of interest for other users?
>
> Yes, it's a quite useful feature. For years, via my Capture templates, I've been adding a property named :Created: to the properties drawer as follows:
>
> :PROPERTIES:
> :Created:  <2022-03-06 Sun 22:42>
> :END:
>
> Now, in 9.5.2, literally hundreds of entries that formerly appeared on the built-in Agenda views cannot be easily found.


It seems that I'm not the only one using this unintended feature in
previous versions of org, and probably there will be many others who
don't use the latest version of org and have not noticed yet but will
have the same problem when they upgrade.

I think that even if timestamps were never intended to be used inside
property drawers before, the fact that it worked for a long time and
nothing in the documentation suggested otherwise makes it a de facto
feature, even if unintended, and should be preserved.

I've located the line in org-agenda.el responsible of the new behavior,
and the following patch seems to fix it. I suggest it is incorporated
into the repository, maybe with a variable
org-agenda-skip-timestamps-in-properties-drawer defaulting to t if not
everyone agrees.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Patch for allowing timestamps inside property drawers again --]
[-- Type: text/x-diff, Size: 858 bytes --]

From 7a27ecd65aa6fea4240756d773c8efe8c4d506fd Mon Sep 17 00:00:00 2001
From: Ignacio <ignacio.decasso@imdea.org>
Date: Tue, 22 Mar 2022 00:18:05 +0100
Subject: [PATCH] fixed

---
 lisp/org/org-agenda.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org/org-agenda.el b/lisp/org/org-agenda.el
index ae0058e037..796b98201d 100644
--- a/lisp/org/org-agenda.el
+++ b/lisp/org/org-agenda.el
@@ -5732,7 +5732,7 @@ org-agenda-get-timestamps
 		    (org-before-first-heading-p)
 		    (and org-agenda-include-inactive-timestamps
 			 (org-at-clock-log-p))
-                    (not (eq 'timestamp (org-element-type (org-element-context)))))
+                    (not (memq (org-element-type (org-element-context)) '(timestamp node-property))))
 	    (throw :skip nil))
 	  (org-agenda-skip))
 	(let* ((pos (match-beginning 0))
-- 
2.25.1

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Samuel Wales]

this is merely intuition, but i can

- understand why planning line tses should not be matched in various
places; they are closely tied to real org entries
- not quite grasp why bare active and inactive tses should not be
matched by ts agenda almost anywhere including blocks.  perhaps the
idea is that you might wnt to store lots of data with tses and they
would be too much clutter?
- understand that there are probably plenty of implementation issues
that might obtrude and should be respected

so i guess i am interested in the rationale.  example of ts [and text]
search being useful might be an example or ledger block that contains
ledger source, or something like that.  i can get why bare ts not
being matched inside links might be useful.

otoh i can get why text [not ts] and isearch search inside of links
can be useful.

idk if my intuitions match those of others.  i am partly trying to
think of what a newcomer might expect to work [and the simplicity of a
rule that he or she would have to remember in order to know what the
behavior will be].


On 3/21/22, Ignacio Casso <ignaciocasso@hotmail.com> wrote:
>
>>> What you see in the new Org version is not a bug. Property values are
>>> treated as plain text by Org.
>>>
>>> I was unable to find a place in manual describing that timestamps cannot
>>> be placed inside property values:
>
>>> I personally see allowing timestamps (and links) inside property values
>>> as a useful feature.
>>> Would it be of interest for other users?
>>
>> Yes, it's a quite useful feature. For years, via my Capture templates,
>> I've been adding a property named :Created: to the properties drawer as
>> follows:
>>
>> :PROPERTIES:
>> :Created:  <2022-03-06 Sun 22:42>
>> :END:
>>
>> Now, in 9.5.2, literally hundreds of entries that formerly appeared on the
>> built-in Agenda views cannot be easily found.
>
>
> It seems that I'm not the only one using this unintended feature in
> previous versions of org, and probably there will be many others who
> don't use the latest version of org and have not noticed yet but will
> have the same problem when they upgrade.
>
> I think that even if timestamps were never intended to be used inside
> property drawers before, the fact that it worked for a long time and
> nothing in the documentation suggested otherwise makes it a de facto
> feature, even if unintended, and should be preserved.
>
> I've located the line in org-agenda.el responsible of the new behavior,
> and the following patch seems to fix it. I suggest it is incorporated
> into the repository, maybe with a variable
> org-agenda-skip-timestamps-in-properties-drawer defaulting to t if not
> everyone agrees.
>
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

RE: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Tom Davey]

Ignacio writes: 

> I've located the line in org-agenda.el responsible of the new behavior,
> and the following patch seems to fix it. I suggest it is incorporated
> into the repository, maybe with a variable org-agenda-skip-timestamps-
> in-properties-drawer defaulting to t if not everyone agrees.

I second that suggestion for the repository! Thanks very much for the patch.

I think you are correct in supposing that when Emacs 28.1 is released, many
Org users who upgrade will be mystified at the new timestamp behavior and
will spend time without success trying to figure out what changed.

Perhaps the new variable you propose,
org-agenda-skip-timestamps-in-properties-drawer, should default to nil to
preserve the historical behavior? 

--
Tom Davey
tom@tomdavey.com
New York NY USA

-----Original Message-----
From: Emacs-orgmode <emacs-orgmode-bounces+tom=tomdavey.com@gnu.org> On
Behalf Of Ignacio Casso
Sent: Monday, March 21, 2022 7:21 PM
To: tom@tomdavey.com
Cc: emacs-orgmode@gnu.org
Subject: Re: [BUG] Agenda no longer works for timestamps inside properties
drawer [9.5.2 (release_9.5.2-24-g668205 @
/home/ignacio/repos/emacs/lisp/org/)]


>> What you see in the new Org version is not a bug. Property values are 
>> treated as plain text by Org.
>>
>> I was unable to find a place in manual describing that timestamps 
>> cannot be placed inside property values:

>> I personally see allowing timestamps (and links) inside property values
as a useful feature.
>> Would it be of interest for other users?
>
> Yes, it's a quite useful feature. For years, via my Capture templates,
I've been adding a property named :Created: to the properties drawer as
follows:
>
> :PROPERTIES:
> :Created:  <2022-03-06 Sun 22:42>
> :END:
>
> Now, in 9.5.2, literally hundreds of entries that formerly appeared on the
built-in Agenda views cannot be easily found.


It seems that I'm not the only one using this unintended feature in previous
versions of org, and probably there will be many others who don't use the
latest version of org and have not noticed yet but will have the same
problem when they upgrade.

I think that even if timestamps were never intended to be used inside
property drawers before, the fact that it worked for a long time and nothing
in the documentation suggested otherwise makes it a de facto feature, even
if unintended, and should be preserved.

I've located the line in org-agenda.el responsible of the new behavior, and
the following patch seems to fix it. I suggest it is incorporated into the
repository, maybe with a variable
org-agenda-skip-timestamps-in-properties-drawer defaulting to t if not
everyone agrees.

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ihor Radchenko]

Ignacio Casso <ignaciocasso@hotmail.com> writes:

>>> What you see in the new Org version is not a bug. Property values are
>>> treated as plain text by Org.
>
> I think that even if timestamps were never intended to be used inside
> property drawers before, the fact that it worked for a long time and
> nothing in the documentation suggested otherwise makes it a de facto
> feature, even if unintended, and should be preserved.

After further reading the source code, I figured that agenda is, in
fact, supposed to handle timestamps inside property drawers. Optional
arguments for org-at-timestamp-p imply that, in agenda specifically,
timestamps inside node properties are considered timestamps despite they
are not being parsed as timestamps by org-element.

> I've located the line in org-agenda.el responsible of the new behavior,
> and the following patch seems to fix it. I suggest it is incorporated
> into the repository, maybe with a variable
> org-agenda-skip-timestamps-in-properties-drawer defaulting to t if not
> everyone agrees.
> -                    (not (eq 'timestamp (org-element-type (org-element-context)))))
> +                    (not (memq (org-element-type (org-element-context)) '(timestamp node-property))))

I pushed a different version of the patch using org-at-timestamp-p to
bugfix as d9bf64f06.

Best,
Ihor

Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)])

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> After further reading the source code, I figured that agenda is, in
> fact, supposed to handle timestamps inside property drawers. Optional
> arguments for org-at-timestamp-p imply that, in agenda specifically,
> timestamps inside node properties are considered timestamps despite they
> are not being parsed as timestamps by org-element.

Even though I fixed the reported issue with agenda not showing headings
with matching timestamps inside property drawers, this situation is
revealing a big inconsistency in Org mode's handling of timestamps.

org-at-timestamp-p usage implies that Org syntax for timestamps is not
only context-dependent, but also depends on current command!

org-at-timestamp-p is called with non-nil argument in a number of
functions in Org:
- org-clock-timestamps-change
- org-mouse-delete-timestamp
- org-mouse-context-menu
- org-follow-timestamp-link
- org-get-repeat
- org-auto-repeat-maybe
- org-time-stamp
- ... many more in org.el

So, depending on the current command, Org may on may not treat objects
matching org-ts-regexp-both as timestamps.

This situation complicates syntax and makes org-element unreliable when
dealing with Org buffers.

Should we just simply allow timestamps to be a part of node property
values? Should we _not_ treat timestamp-looking text outside their
allowed contexts (like quotes, source blocks, etc) as timestamps?

Best,
Ihor

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ihor Radchenko]

Samuel Wales <samologist@gmail.com> writes:

> - not quite grasp why bare active and inactive tses should not be
> matched by ts agenda almost anywhere including blocks.  perhaps the
> idea is that you might wnt to store lots of data with tses and they
> would be too much clutter?

See
https://orgmode.org/list/20220101122409.GA29829@itccanarias.org

Best,
Ihor

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ihor Radchenko]

Samuel Wales <samologist@gmail.com> writes:

> so i guess i am interested in the rationale.  example of ts [and text]
> search being useful might be an example or ledger block that contains
> ledger source, or something like that.  i can get why bare ts not
> being matched inside links might be useful.
>
> otoh i can get why text [not ts] and isearch search inside of links
> can be useful.
>
> idk if my intuitions match those of others.  i am partly trying to
> think of what a newcomer might expect to work [and the simplicity of a
> rule that he or she would have to remember in order to know what the
> behavior will be].

I can see your point. Simple timestamp matching by regexp disregarding
timestamp objects might be useful. However, I do not think that matching
should be via timestamps in such a case. Rather it should be even more
lax - agenda may match anything date-looking (with or without brackets).
A specialized regex agenda matcher with regex constructed using current
agenda date.

Maybe we can introduce a new agenda view? Or maybe a special agenda
mode, similar to org-agenda-include-inactive-timestamps?

Best,
Ihor

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Samuel Wales <samologist@gmail.com> writes:
>
>> so i guess i am interested in the rationale.  example of ts [and text]
>> search being useful might be an example or ledger block that contains
>> ledger source, or something like that.  i can get why bare ts not
>> being matched inside links might be useful.
>>
>> otoh i can get why text [not ts] and isearch search inside of links
>> can be useful.
>>
>> idk if my intuitions match those of others.  i am partly trying to
>> think of what a newcomer might expect to work [and the simplicity of a
>> rule that he or she would have to remember in order to know what the
>> behavior will be].
>
> I can see your point. Simple timestamp matching by regexp disregarding
> timestamp objects might be useful. However, I do not think that matching
> should be via timestamps in such a case. Rather it should be even more
> lax - agenda may match anything date-looking (with or without brackets).
> A specialized regex agenda matcher with regex constructed using current
> agenda date.
>
> Maybe we can introduce a new agenda view? Or maybe a special agenda
> mode, similar to org-agenda-include-inactive-timestamps?
>

Perhaps I simply don't understand, but I fail to see the need for adding
such functionality. Org files are plain text and you can just use the
build-in or any of the add-on search tools to search for anything,
including timestamps or things which may look like timestamps.

The only time org needs to provide search functionality is when that
search does need to be done within the context of a date/time object.

Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)])

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> After further reading the source code, I figured that agenda is, in
>> fact, supposed to handle timestamps inside property drawers. Optional
>> arguments for org-at-timestamp-p imply that, in agenda specifically,
>> timestamps inside node properties are considered timestamps despite they
>> are not being parsed as timestamps by org-element.
>
> Even though I fixed the reported issue with agenda not showing headings
> with matching timestamps inside property drawers, this situation is
> revealing a big inconsistency in Org mode's handling of timestamps.
>
> org-at-timestamp-p usage implies that Org syntax for timestamps is not
> only context-dependent, but also depends on current command!
>
> org-at-timestamp-p is called with non-nil argument in a number of
> functions in Org:
> - org-clock-timestamps-change
> - org-mouse-delete-timestamp
> - org-mouse-context-menu
> - org-follow-timestamp-link
> - org-get-repeat
> - org-auto-repeat-maybe
> - org-time-stamp
> - ... many more in org.el
>
> So, depending on the current command, Org may on may not treat objects
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable when
> dealing with Org buffers.
>
> Should we just simply allow timestamps to be a part of node property
> values? Should we _not_ treat timestamp-looking text outside their
> allowed contexts (like quotes, source blocks, etc) as timestamps?
>

I think we have to be very wary here. I can see any changes here causing
lots of breakage for people. I know for my own use case, I use
timestamps a lot in property draws and various source blocks. I never
want any of them showing up in my agenda.

As an example, I was recently working for a company which required that
you put a timestamp in both a file header and in comments. The format
they used was pretty much the same as an org-mode active timestamp. I
use org mode to tangle the source files I write (as well as manage my
client data, such as todos, invoicing, contacts etc), so these files are
searched for agenda items, but I do not want any of those timestamps
causing lines in my agenda views. These timestamps are most commonly
found in source and example blocks.

I think the only time an org timestamp should be recognised in a source
block is when that source block is an org-mode source block. I don't
think they should ever be 'recognised' in example blocks.  

IN addition, my invoicing solution, which is based on org, uses
timestamps to track invoice periods etc. None of these should ever
appear in the agenda. This information is typically tracked in property
draws.

Unfortunately, I think org timestamps is a bit of a mess and we need to
be very careful about further complicating matters. The inability to
handle timezones is a major limitation IMO. The inability to handle
daylight savings transitions in a consistent manner (particularly for
calculation of periods, duration, etc) is often a source of errors and
if you are unfortunate enough to regularly travel across different
timezones, forget about using org mode to manage your schedule.

I have done several deep dives into org-mode's timestamp stuff, but
usually come back up gasping for air. Managing data and time data is
often far more complicated than it may appear on the surface. I think we
need to be extremely conservative when considering changes in this area
(it is the main reason I've never managed to find a way to add time zone
data - every solution I could think of was either really high on the
level of breakage and frustration it would create for users or it
adversely impacted the convenience/usability of org timestamps). 

Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions

[Nicolas Goaziou]

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> So, depending on the current command, Org may on may not treat objects
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable when
> dealing with Org buffers.

This is orthogonal to syntax. I think the docstring of that predicate is
clear: `org-at-timestamp-p' is a convenience function for broader uses
of timestamps, which existed before Element.

> Should we just simply allow timestamps to be a part of node property
> values? Should we _not_ treat timestamp-looking text outside their
> allowed contexts (like quotes, source blocks, etc) as timestamps?

Allowing Org syntax in property values is creating another set of
problems: often the value is really a string that Org shouldn't try to
interpret.

Regards,
-- 
Nicolas Goaziou

RE: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)])

[Tom Davey]

Hi Tim, 

Thanks for these thoughtful comments. I agree that the Org developers (to
whom I, as a mere user, owe enormous thanks) must be wary before making
changes to how timestamps are handled. 

This argues, I would say, for keeping what I believe was the status quo
since at least Org version 9.4.4: Agenda views would display entries with
active timestamps in property drawers. That has been my historical
experience. 

Tim, has your historical experience been different? In the invoicing example
you give, were the timestamps in the properties drawer active, or inactive? 

I have just verified with a simple test that Org version 9.4.4, which was
shipped with Emacs 27.2 I believe, does display entries with an active
timestamp as the value of a property in the ordinary :PROPERTIES: drawer.
That's the situation I'm calling the "status quo." I'm wondering if my
experience coincides with the experience of others. 

Here's the simple entry that will be shown on the Week/Day Agenda view in
9.4.4: 

* TODO Test of active timestamps
   :PROPERTIES:
   :Created:  <2022-03-22 Tue 18:30>
   :END:

And note this: adding a second active timestamp to the test entry, e.g., to
accompany a SCHEDULED: keyword, results in the entry appearing on the Agenda
twice, as would be expected: 

* TODO Test of active timestamps
   SCHEDULED: <2022-03-22 Tue 18:30>
   :PROPERTIES:
   :Created:  <2022-03-22 Tue 18:30>
   :END:

This second example shows why the variable
org-agenda-skip-additional-timestamps-same-entry is valuable. I rarely want
an entry to display twice on the same day. 

Tom Davey 

--
Tom Davey
tom@tomdavey.com
New York NY USA

-----Original Message-----
From: Emacs-orgmode <emacs-orgmode-bounces+tom=tomdavey.com@gnu.org> On
Behalf Of Tim Cross
Sent: Tuesday, March 22, 2022 5:10 PM
To: Ihor Radchenko <yantar92@gmail.com>
Cc: Ignacio Casso <ignaciocasso@hotmail.com>; emacs-orgmode@gnu.org;
tom@tomdavey.com; Nicolas Goaziou <mail@nicolasgoaziou.fr>
Subject: Re: Timestamp parsing inside node properties and other contexts out
of org-element-object-restrictions (was: [BUG] Agenda no longer works for
timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @
/home/ignacio/repos/emacs/lisp/org/)])


Ihor Radchenko <yantar92@gmail.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> After further reading the source code, I figured that agenda is, in 
>> fact, supposed to handle timestamps inside property drawers. Optional 
>> arguments for org-at-timestamp-p imply that, in agenda specifically, 
>> timestamps inside node properties are considered timestamps despite 
>> they are not being parsed as timestamps by org-element.
>
> Even though I fixed the reported issue with agenda not showing 
> headings with matching timestamps inside property drawers, this 
> situation is revealing a big inconsistency in Org mode's handling of
timestamps.
>
> org-at-timestamp-p usage implies that Org syntax for timestamps is not 
> only context-dependent, but also depends on current command!
>
> org-at-timestamp-p is called with non-nil argument in a number of 
> functions in Org:
> - org-clock-timestamps-change
> - org-mouse-delete-timestamp
> - org-mouse-context-menu
> - org-follow-timestamp-link
> - org-get-repeat
> - org-auto-repeat-maybe
> - org-time-stamp
> - ... many more in org.el
>
> So, depending on the current command, Org may on may not treat objects 
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable 
> when dealing with Org buffers.
>
> Should we just simply allow timestamps to be a part of node property 
> values? Should we _not_ treat timestamp-looking text outside their 
> allowed contexts (like quotes, source blocks, etc) as timestamps?
>

I think we have to be very wary here. I can see any changes here causing
lots of breakage for people. I know for my own use case, I use timestamps a
lot in property draws and various source blocks. I never want any of them
showing up in my agenda.

As an example, I was recently working for a company which required that you
put a timestamp in both a file header and in comments. The format they used
was pretty much the same as an org-mode active timestamp. I use org mode to
tangle the source files I write (as well as manage my client data, such as
todos, invoicing, contacts etc), so these files are searched for agenda
items, but I do not want any of those timestamps causing lines in my agenda
views. These timestamps are most commonly found in source and example
blocks.

I think the only time an org timestamp should be recognised in a source
block is when that source block is an org-mode source block. I don't think
they should ever be 'recognised' in example blocks.  

IN addition, my invoicing solution, which is based on org, uses timestamps
to track invoice periods etc. None of these should ever appear in the
agenda. This information is typically tracked in property draws.

Unfortunately, I think org timestamps is a bit of a mess and we need to be
very careful about further complicating matters. The inability to handle
timezones is a major limitation IMO. The inability to handle daylight
savings transitions in a consistent manner (particularly for calculation of
periods, duration, etc) is often a source of errors and if you are
unfortunate enough to regularly travel across different timezones, forget
about using org mode to manage your schedule.

I have done several deep dives into org-mode's timestamp stuff, but usually
come back up gasping for air. Managing data and time data is often far more
complicated than it may appear on the surface. I think we need to be
extremely conservative when considering changes in this area (it is the main
reason I've never managed to find a way to add time zone data - every
solution I could think of was either really high on the level of breakage
and frustration it would create for users or it adversely impacted the
convenience/usability of org timestamps). 

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Samuel Wales]

On 3/22/22, Ihor Radchenko <yantar92@gmail.com> wrote:
> lax - agenda may match anything date-looking (with or without brackets).
> A specialized regex agenda matcher with regex constructed using current
> agenda date.
>
> Maybe we can introduce a new agenda view? Or maybe a special agenda
> mode, similar to org-agenda-include-inactive-timestamps?

over the years i have seen different opinions on contexts in which
tses [and links, not to make it more complex but to in principle
strive for some type of orthogonality iff discussion goes there]
should show up in agenda and where it should not.

[as a same page type of check, for tses we are talking about
"daily/weekly" which has different semantics from search agenda and
cannot be reproduced with search agenda via parameterization.]

another context [not to create controversy but also for in principle
orthogonality] is line comments, which some think should mean tses and
links therein should not show up in agenda and not fontify and not be
clickable [rule = "remove tses and links from org semantics and
fontification including agenda but not including certain org searchers
like org-occur-in-agend-files"], while others think should mean
"remove from all export" [use case: so you as author can document
exportable stuff inline using comments and still have your tses show
up in daily/weekly if you want that and have clickable links without
having those exposed to the recipient of the exported document, etc.].
links in line comments can be worked around with a bit of code, but
idk about tses.  some will want tses commentable.  some not.

your idea of expanding to other date-like things is an interesting
idea, and so is making an analogy with log mode.

another possibility is to satisfy the preferences users have expressed
[and those with n>1 needs] using a single variable that contains the
contexts that should show in daily/weekly agenda.  org uses multi-item
variables more frequently in recent years [e.g.
org-show-context-detail or visibility for sparse trees] which reduces
vars.  you might have had log mode items in mind, which is similar.

thus in principle it need not be a mode like log mode items, but could
be in agenda custom commnds.

i think a whole view [unlike log mode and tag filtering] might be
confusing to newcomers as [just as with sorting and log mode items and
allow COMMENT and allow ARCHIVE] the regular daily/weekly view is
parameterizable.

i know i got confused by the todo view, wondering if it was something
not creatable with parameterization.

so i would go for your second idea, or the var.  i do think different
preferences exist out there, some strong.

with one or the other, future defaults are trivially changed,
transparent/discoverble by user, and open to user change.

[one problem now being that users can be not merely surprised, but
unaware that stuff is missing from agenda.]

>
> Best,
> Ihor
>

Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)])

[Tim Cross]

"Tom Davey" <tom@tomdavey.com> writes:

> Hi Tim, 
>
> Thanks for these thoughtful comments. I agree that the Org developers (to
> whom I, as a mere user, owe enormous thanks) must be wary before making
> changes to how timestamps are handled. 
>
> This argues, I would say, for keeping what I believe was the status quo
> since at least Org version 9.4.4: Agenda views would display entries with
> active timestamps in property drawers. That has been my historical
> experience. 
>
> Tim, has your historical experience been different? In the invoicing example
> you give, were the timestamps in the properties drawer active, or inactive? 
>
> I have just verified with a simple test that Org version 9.4.4, which was
> shipped with Emacs 27.2 I believe, does display entries with an active
> timestamp as the value of a property in the ordinary :PROPERTIES: drawer.
> That's the situation I'm calling the "status quo." I'm wondering if my
> experience coincides with the experience of others. 
>
> Here's the simple entry that will be shown on the Week/Day Agenda view in
> 9.4.4: 
>
> * TODO Test of active timestamps
>    :PROPERTIES:
>    :Created:  <2022-03-22 Tue 18:30>
>    :END:
>
> And note this: adding a second active timestamp to the test entry, e.g., to
> accompany a SCHEDULED: keyword, results in the entry appearing on the Agenda
> twice, as would be expected: 
>
> * TODO Test of active timestamps
>    SCHEDULED: <2022-03-22 Tue 18:30>
>    :PROPERTIES:
>    :Created:  <2022-03-22 Tue 18:30>
>    :END:
>
> This second example shows why the variable
> org-agenda-skip-additional-timestamps-same-entry is valuable. I rarely want
> an entry to display twice on the same day. 
>

For my invoicing system, the timestamps are in property draws, but not
under TODO headings, so I guess they wouldn't show up. I have used
timestamps in TODO entry property draws, but don't recall seeing them in
the agenda when I do. However, I also use a couple of custom agenda
views more often than the default agenda view, so I just may not have
noticed.

Personally, I have no use case for TODOs being displayed in the agenda
based on their created date. I do like to log when I created the todo,
but would not want to see that cluttering my agenda where I only want to
see scheduled and deadline tasks. I use a property draw for this type of
information as I consider it metadata about the entry. If I really want
ot see the item in the agenda based on a timestamp, then I explicitly
add that timestamp to the entry (not in  a property draw).

In my invoicing module, property draws are used to track invoicing
metadata i.e. last invoice number, last invoice date, invoice creation
date and invoice period for each invoice, invoice due date, paid datge
etc. Some of the timestamps are 'inactive' (they never change) and some
are active (will get modified). Perhaps that was a mistake on my part
and all of them should be inactive, but at the time, I thought of the
distinction as not being purely about whether they show in the agenda,
but more about whether they were immutable or not. 

The need to have another variable to limit the scope of timestamps when
you do allow timestamps in property draws makes things feel very
'kludgy'. I get very concerned about all these 'enhancements' we keep
adding which then also require additional complexity with multiple
variables interacting at different levels. I find the number of
variables and complexity associated with the agenda compared to when I
first started using org mode to be mind boggling. I often wonder how the
maintainers are able to keep on top of the increasing complexity of org
mode. 

What I definitely would NOT want is for TODO tasks in source blocks
which have a timestamp associated with them showing up in the agenda.
The possible exception might be when the source block is an org source
block. What I want in source blocks is for them to behave like a
microcosm of the original code environment - I want them to behave
exactly as they would when I just edit that code in a code dedicated
buffer with the addition of a minimal set of babel/noweb features to
allow me to work with code spread over multiple source code blocks etc.
I definitely don't want the additional processing overhead of org mode
search through my source blocks looking for things which look like they
might be timestamps or worse, looking for such things and then trying to
do something clever with them (like adding to the agenda).

With regards to timestamps in property draws, I'm not sure that arguing
the status quo is to keep that behaviour. I'm not sure that behaviour
was supposed to work or that it is well defined. It may have worked
previously 'by accident' rather than design (something which tends to
happen more as complexity increases). For example, what is the situation
with timestamps in property draws and inheritance? What is the situation
with timestamps and property draws for items which are not TODOs (i.e.
normal headings with a property draw)? What happens if I have multiple
properties with timestamps as their value?

I would agree with Ihor's observation things seem to be inconsistent and
we probably need to work at getting things to be more consistent.
However, I would lean towards a solution which simplifies the situation
in the general case, even if that makes some specialised uses cases more
difficult. Given that property draws can have arbitrary property names
and some of these could be active timestamps and given there could be
multiple properties which have timestamp values in the same property
draw and there is supposed to be an inheritance component to properties,
I would be inclined to not support adding properties with timestamps to
the agenda. If it turns out a lot of people want this functionality,
then I would suggest a specific property name be reserved for this i.e.
:Agenda-ts: or :Agenda-entry:, rather than allowing any property name with a timestamp. 
I wouldn't use :Created: as that would be overloading the property to
perform two roles (track when an entry was created and add it to the
agenda).

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> Perhaps I simply don't understand, but I fail to see the need for adding
> such functionality. Org files are plain text and you can just use the
> build-in or any of the add-on search tools to search for anything,
> including timestamps or things which may look like timestamps.
>
> The only time org needs to provide search functionality is when that
> search does need to be done within the context of a date/time object.

That's exactly the context I was referring to. It is impossible to
construct a regexp matcher for timestamps matching current day in
agenda, especially when user can dynamically jump to unknown date using
"j".

Best,
Ihor

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Ihor Radchenko]

Samuel Wales <samologist@gmail.com> writes:

> your idea of expanding to other date-like things is an interesting
> idea, and so is making an analogy with log mode.
>
> another possibility is to satisfy the preferences users have expressed
> [and those with n>1 needs] using a single variable that contains the
> contexts that should show in daily/weekly agenda.  org uses multi-item
> variables more frequently in recent years [e.g.
> org-show-context-detail or visibility for sparse trees] which reduces
> vars.  you might have had log mode items in mind, which is similar.

Your idea about multi-term variables reminds me about
org-agenda-log-mode-items. So, the proposed broad matching of anything
time stamp-like under headlines might be a new option in that variable:

org-agenda-log-mode-items is a variable defined in org-agenda.el.
Documentation
List of items that should be shown in agenda log mode.

This list may contain the following symbols:

  closed    Show entries that have been closed on that day.
  clock     Show entries that have received clocked time on that day.
  state     Show all logged state changes.
Note that instead of changing this variable, you can also press C-u l in
the agenda to display all available LOG items temporarily.

> [one problem now being that users can be not merely surprised, but
> unaware that stuff is missing from agenda.]

To clarify, this bug report came after another commit I introduced. That
commit fixed a user complaint that it was literally impossible to
prevent headlines from being shown in agenda when, for example, a
timestamp was inside a code block or quote block.
https://orgmode.org/list/20220101122409.GA29829@itccanarias.org

So, in the past, agenda showed every headline containing matching active
(or inactive, if inactive timestamps where set to be included in agenda)
timestamp anywhere under headline regardless of the context.
I changed that, causing the bug report here.

Now, I fixed the regression noticing that agenda views where intended to
catch timestamps inside property drawers according to
(org-at-timestamp-p 'agenda)

The current fix did not restore the initial behaviour of including every
possible timestamp regardless of the context, but that behaviour appears
to be unintentional given the docstring of org-at-timestamp-p:

Non-nil if point is inside a timestamp.

By default, the function only consider syntactically valid active
timestamps.  However, the caller may have a broader definition
for timestamps.  As a consequence, optional argument EXTENDED can
be set to the following values

...
  agenda

    Include timestamps allowed in Agenda, i.e., those in
    properties drawers, planning lines and clock lines.

> another context [not to create controversy but also for in principle
> orthogonality] is line comments, which some think should mean tses and
> links therein should not show up in agenda and not fontify and not be
> clickable [rule = "remove tses and links from org semantics and
> fontification including agenda but not including certain org searchers
> like org-occur-in-agend-files"], while others think should mean
> "remove from all export" [use case: so you as author can document
> exportable stuff inline using comments and still have your tses show
> up in daily/weekly if you want that and have clickable links without
> having those exposed to the recipient of the exported document, etc.].
> links in line comments can be worked around with a bit of code, but
> idk about tses.  some will want tses commentable.  some not.
>

Could you elaborate? Maybe provide an example.

Best,
Ihor

Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions

[Ihor Radchenko]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> So, depending on the current command, Org may on may not treat objects
>> matching org-ts-regexp-both as timestamps.
>>
>> This situation complicates syntax and makes org-element unreliable when
>> dealing with Org buffers.
>
> This is orthogonal to syntax. I think the docstring of that predicate is
> clear: `org-at-timestamp-p' is a convenience function for broader uses
> of timestamps, which existed before Element.

Let me clarify. What I have in mind is my proposal about using
org-element for fontification:
https://orgmode.org/list/87ee7c9quk.fsf@localhost

The usage org `org-at-timestamp-p' in multiple places in org.el implies
that Org treats timestamp-like strings as actual timestamps even when
org-element does not recognise them. Then, if, say, I implement a new
fontification system purely relying on org-element, some timestamp-like
strings will remain interactive (for example, using mouse context menu)
but will not be fontified.

Any other idea relying on org-element might also suffer from such
issues.

>> Should we just simply allow timestamps to be a part of node property
>> values? Should we _not_ treat timestamp-looking text outside their
>> allowed contexts (like quotes, source blocks, etc) as timestamps?
>
> Allowing Org syntax in property values is creating another set of
> problems: often the value is really a string that Org shouldn't try to
> interpret.

Is there an easy example demonstrating the potential problem?
For reference, I did try to implement parsing node-property values and
even did not fail any tests, except the one directly checking the
current timestamp-in-property-drawer parsing.

Best,
Ihor

Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)])

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> I think we have to be very wary here. I can see any changes here causing
> lots of breakage for people. I know for my own use case, I use
> timestamps a lot in property draws and various source blocks. I never
> want any of them showing up in my agenda.

FYI, the default agenda behaviour around a month ago was exactly what
you want to avoid (assuming that you use active timestamps). I explained
in more details in https://list.orgmode.org/87h77psaik.fsf@gmail.com/T/#me4f41a8b36ada384d87cf028ef10164dfa526ad2
Let's keep discussion of the original issue in the original thread.

> Unfortunately, I think org timestamps is a bit of a mess and we need to
> be very careful about further complicating matters. The inability to
> handle timezones is a major limitation IMO. The inability to handle
> daylight savings transitions in a consistent manner (particularly for
> calculation of periods, duration, etc) is often a source of errors and
> if you are unfortunate enough to regularly travel across different
> timezones, forget about using org mode to manage your schedule.
>
> I have done several deep dives into org-mode's timestamp stuff, but
> usually come back up gasping for air. Managing data and time data is
> often far more complicated than it may appear on the surface. I think we
> need to be extremely conservative when considering changes in this area
> (it is the main reason I've never managed to find a way to add time zone
> data - every solution I could think of was either really high on the
> level of breakage and frustration it would create for users or it
> adversely impacted the convenience/usability of org timestamps). 

I sympathise with you. I still remember my struggle when I had to travel
across multiple time zones. Anyway, the timezone issue have been
discussed multiple times within last 10 years or so. It is complex and
nobody managed to come with a good idea how to approach it.

Best,
Ihor

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

[Samuel Wales]

***** x2
> Could you elaborate? Maybe provide an example.

idk if this answers your q or not.

it rambles.

for context some goals include orthogonality, satisfying
strong differing views/needs with little complexity, and
having simple rules or transparency for knowing behavior
e.g. what shows up in an agenda view.

by line comments i mean "# " in recent org since 9 or so and
"#" in org up until then.  with leading spaces or whatever.

for clarity i will treat COMMENT quasi-kw separately if at
all.  it is like ARCHIVE.

fundamentally line comments prevent lines from being
exported.  so you could do

\* my header to be exported via subtree export
# NOTE TO SELF: see https://google.com for more on this.
# fixme see https://whatever.whatever for new change
something that will actually export

and that was just for yourself.  it would not get exported.
for reference.

and the link would be for this case highlighted and
clickable [and show up in agenda text view].

idr the details now, but links were clickable without being
highlighted at one point, and accidental clicks were
dangerous.  you could work around that with font lock and
idk what changes have occurred since then.

i have just described the "line comments mean don't show in
export" rule.  the intuition of some has been that links
should as above still be useful, even though they do not
export.  comenting is mostly related to exporting.

others have thought commenting means commenting.  they think
there should be no semantics at all when you comment links.
thus, links are inert pieces of text.  if you want to go to
google then you copy the text and paste it into a browser.

===

up to here i have described links.  but ts issues are
similar to links.  e.g. fontifying and clicking.

they are supposed to show up in daily/weekly for the date of
the ts.  except when they are not supposed to.  which varies
by preference.  (which is reolated to the currentish
controversyish wrt drawers and blocks --- i am saying line
comments are a valid similar consideration.  i have a
possible solution for all.)

for a non-exporting entry, you might want to comment out a
ts so it does not show up.  what is canonical there?  the
second type of user says just comment it.  the first type of
user says tses should be useful in comments.

\* my header to be exported via subtree export
# i wrote this paragraph on this date: [2022-03-23 Wed]
something that will actually export

or the active ts version.  tses are also highly useful to
see fontified and be clickable analogously to links.  (to
the first type of user.)  you might want to know the changes
you made to your docuyment on that date, so it shows in
daily/weekly agenda [stndard disclaimes like in log mode if
inactive etc.].  you should not be restricted to text search
to find a ts or a range of tses.

you want things to be able to show up nd still not be
exported.

some would say if it is fontified, you know it is a ts; it's
visually useful even if you do not use the semantics.

some might make the point that fundamentally active tses are
meant to show up by default in daily/weekly and if you want
something to not show up in non-log-mode you should just
make it inactive.

you could consider daily/weekly being different from text
search to be an inconsistency which requires fixing by
making it transparent, one possibility being making it in a
variable saying where tses will show up.

===

so to a solution for most of this.  i rambled and idk what
you are asking for.

first, i made a mistake due to brain not working, in my
parenthetical examples, but i think you got the idea.

just to get that part nailed down even though it was
probably obvious, vars can contain sets.  thus, a single var
can dictate e.g. what shows up in daily/weekly agenda.  if a
ts is in a context that is mentioned in a member of var,
then it shows up in daily/weekly agenda view.  the var's
value could be e.g. '(properties-drawers line-comments).
(perhaps modulo some reasonable thing to do for body text,
headers, etc. where it is not worth having them be
explicitly members in that var or so.)

thus if some users want tses in properties drawers (or
source blocks, or line comments) to show up in daily/weekly
and other users do not, then this might be a solution for
satisfying both types of users, and also those who have n>1
use case (sometimes one and soetimes the other).  and
changing default would be trivial and user can change it and
user can c-h v thus the user does not have to guess about
behavior or think about org versions or pull hair out trying
to comment something out from showing up or GET it to show
up.

note that this reduces a bunch of potential variables down
to just one.  merely: what shows up in daily/weekly view.

for some purposes, you cn sticik this variable in a custom
command clause.

for completeness, other considerations exist like
clickability (viz. in what contexts should a ts or link be
clickable?  perhaps all thus no need to specify in a var?),
fontifying (perhaps simplest rule is "if a ts or link is
clickable, it should be fontified and vice-versa" with no
need to specify in a var?), what should show up in text
search view (an analogous var or not necessary because
everything shows up?) and org-occur-in-agenda-files
(currently inconsistent with the second type of user) and
isearch --- personally it drives me crazy tht parts of links
are not isearchable without doing visible-mode first but
that is just me.  there are inconsistencies of various
strengths.  some of them probably fixable while satisfying
desieratda if there is a desire to.