Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Luke Amdor]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

Given a test.org with:

* [#B] Test
** TODO one


When I do the agenda to find TODO headlines with priority B I would expect
it to find the "TODO one", however it doesn't. I'm very sure this regressed
when upgrading to 8.3.1, as it worked fine in 8.3.0. Thanks!


Emacs  : GNU Emacs 24.5.1 (x86_64-apple-darwin14.1.0, NS
apple-appkit-1344.72)
 of 2015-04-17 on MacBook-Pro.local
Package: Org-mode version 8.3.1 (8.3.1-56-g17a225-elpa @
/Users/luke/.emacs.d/elpa/org-20150817/)

(add-to-list 'load-path (expand-file-name "./org-mode/lisp"))
(add-to-list 'load-path (expand-file-name "./org-mode/contrib/lisp" t))

(setq inhibit-startup-message t)

(setq org-use-property-inheritance t)

(setq org-agenda-files (list "test.org"))

(setq org-agenda-custom-commands
      '(("t" "Test" ((tags-todo "+PRIORITY=\"B\"")))))


Luke

[-- Attachment #2: Type: text/html, Size: 1389 bytes --]

Re: Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Nicolas Goaziou]

Hello,

Luke Amdor <luke.amdor@gmail.com> writes:

> Given a test.org with:
>
> * [#B] Test
> ** TODO one
>
>
> When I do the agenda to find TODO headlines with priority B I would expect
> it to find the "TODO one", however it doesn't. I'm very sure this regressed
> when upgrading to 8.3.1, as it worked fine in 8.3.0. Thanks!

There is a misunderstanding here. Special properties, like PRIORITY,
cannot be inherited.

However, in this case, "one" has a default priority, defined in
`org-default-priority'. Since it is usually B, it should appear
nonetheless in your sparse tree. 

It is fixed. Thank you.


Regards,

-- 
Nicolas Goaziou

Re: Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Bastien]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> There is a misunderstanding here. Special properties, like PRIORITY,
> cannot be inherited.

The PRIORITY property *was* inherited.  At least in 8.2.10.

It is reasonable to expect the PRIORITY property to be inherited and
the new behavior (falling back on the default priority) is confusing.

I see a lot of work on the way Org get properties since one year.

How hard would it be to restore the previous behavior?

Are there any other changes that we need to know from a user point of
view?

Thanks,

-- 
 Bastien

Re: Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> There is a misunderstanding here. Special properties, like PRIORITY,
>> cannot be inherited.
>
> The PRIORITY property *was* inherited.  At least in 8.2.10.

AFAIK, PRIORITY never was intended to be inherited. Inheritance applies
to node properties, i.e. property drawers, whereas special properties,
like PRIORITY are, by definition, not set through property drawers.

There is also a technical issue: Org defines a default priority, so
PRIORITY is never empty. Again, inheritance kicks in when a property is
undefined at some level. This never happens in this case.

Regards,

Re: Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Nicolas Goaziou]

Correcting myself,

> AFAIK, PRIORITY never was intended to be inherited. Inheritance applies
> to node properties, i.e. property drawers, whereas special properties,
> like PRIORITY are, by definition, not set through property drawers.

This is incorrect. What I mean is inheritance is not automatic for
special properties, unlike to regular node properties.

In any case, this doesn't depend on `org-use-property-inheritance'. Few
of the special properties are inherited, e.g., BLOCKED, and /always/
are, most are never inherited, e.g. ITEM.

> There is also a technical issue: Org defines a default priority, so
> PRIORITY is never empty. Again, inheritance kicks in when a property is
> undefined at some level. This never happens in this case.

We need to redefine `org-default-prority' to solve this, e.g., the
variable only applies to top-level items.

Re: Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Bastien]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Correcting myself,
>
>> AFAIK, PRIORITY never was intended to be inherited. Inheritance applies
>> to node properties, i.e. property drawers, whereas special properties,
>> like PRIORITY are, by definition, not set through property drawers.
>
> This is incorrect. What I mean is inheritance is not automatic for
> special properties, unlike to regular node properties.
>
> In any case, this doesn't depend on `org-use-property-inheritance'. Few
> of the special properties are inherited, e.g., BLOCKED, and /always/
> are, most are never inherited, e.g. ITEM.
>
>> There is also a technical issue: Org defines a default priority, so
>> PRIORITY is never empty. Again, inheritance kicks in when a property is
>> undefined at some level. This never happens in this case.
>
> We need to redefine `org-default-prority' to solve this, e.g., the
> variable only applies to top-level items.

After a few tests, I'm confused and I don't understand all the changes
in this area (and they are not documented in ORG-NEWS.)

Let's forget about what "special" means and how properties are
displayed and set in the buffer.

We have four categories of properties:

1. those which are *never* inherited (like ITEM)
2. those which are *always* inherited (like CATEGORY)
3. those which inheritance relies on `org-use-property-inheritance'
4. those which are not part of the previous types

(4) sounds a bit borgesian, but it's important: my understanding is
that this fourth category *should* be empty.  Apparently it is not
empty, since the PRIORITY property inheritance is not controlled by
`org-use-property-inheritance'.  Are there other exceptions?

There is a default priority the same way there is a default category:
having a default value for the PRIORITY property should not prevent
inheriting it from a superior headline IMO, and the previous behavior
was right.

I know dwelling on this is boring but we need to do so, at least to
clarify changes in etc/ORG-NEWS.

Thanks,

-- 
 Bastien

Re: Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

> After a few tests, I'm confused and I don't understand all the changes
> in this area (and they are not documented in ORG-NEWS.)

I'd rather clarify something here.

Most changes introduced come from bug reports, i.e., they are not a will
to change how things work in Org. Fixed bugs are not documented in
ORG-NEWS.

Unfortunately, once the fix is applied, it is hard to know if it
actually modifies some behaviour because the documentation doesn't, and
cannot, explain all the details and there are no tests to get a clue
about the original intent. For core functions, the bug fix usually
includes specifications tests to avoid encountering the same issue
again.

This is exactly what happened with "COMMENT" headlines. This is almost
the same for property API (the current change): I wanted to improve it
but had to switch from an ad-hoc model to a structured one.

To sum it up, this is not carelessness in filling up ORG-NEWS (though
omissions do happens occasionally) but fuzziness in how features are
described.

> Let's forget about what "special" means and how properties are
> displayed and set in the buffer.

On the contrary, it is important to understand that a few properties do
not follow the regular model and need to be handled specifically.

However, me way extend what "special" means. At the moment it only
includes properties not set through a property drawer. We could include
properties with a hard-coded inheritance patters (i.e., CATEGORY and
PRIORITY).

> We have four categories of properties:
>
> 1. those which are *never* inherited (like ITEM)
> 2. those which are *always* inherited (like CATEGORY)
> 3. those which inheritance relies on `org-use-property-inheritance'
> 4. those which are not part of the previous types
>
> (4) sounds a bit borgesian, but it's important: my understanding is
> that this fourth category *should* be empty.

I agree.

> Apparently it is not empty, since the PRIORITY property inheritance is
> not controlled by `org-use-property-inheritance'. Are there other
> exceptions?

I cannot think of any.

> There is a default priority the same way there is a default category:
> having a default value for the PRIORITY property should not prevent
> inheriting it from a superior headline IMO, and the previous behavior
> was right.

You are right, but I'm pretty sure the previous implementation of the
property API didn't implement it this way, i.e., PRIORITY was not
implemented like CATEGORY, at all. My gut feeling is that PRIORITY
inheritance happened by side-effect.

In any case, I see no objection to treat PRIORITY much like CATEGORY.


Regards,

Re: Bug: Priorities not inheriting [8.3.1 (8.3.1-56-g17a225-elpa @ /Users/luke/.emacs.d/elpa/org-20150817/)]

[Nicolas Goaziou]

Completing myself:

> You are right, but I'm pretty sure the previous implementation of the
> property API didn't implement it this way, i.e., PRIORITY was not
> implemented like CATEGORY, at all. My gut feeling is that PRIORITY
> inheritance happened by side-effect.
>
> In any case, I see no objection to treat PRIORITY much like CATEGORY.

After some investigations, it appears that priority handling prior to
the changes (I tested in Org 7.9.4) was very confusing.

It appears "PRIORITY" could be inherited but not up to
`org-default-priority'. In the following example

  * [#A] Top
  ** Test 1
  * Top
  ** Test 2

evaluating (org-entry-get (point) "PRIORITY" t) on second line returned
"A", but nil on "Test 2". I would have expected "B" (i.e.,
`org-default-priority', as a string).

However, (org-entry-get (point) "PRIORITY") is never used throughout the
code base. Instead, Org relies on alternates solutions.

For example, `org-sort-entries' uses the following snippet, for priority
sorting:

  ((= dcst ?p)
   (if (re-search-forward org-priority-regexp (point-at-eol) t)
       (string-to-char (match-string 2))
     org-default-priority))

which also completely ignores inheritance. Sometimes, a dedicated
function, `org-get-priority', is used:

  (defun org-get-priority (s)
    "Find priority cookie and return priority."
    (save-match-data
      (if (functionp org-get-priority-function)
  	(funcall org-get-priority-function)
        (if (not (string-match org-priority-regexp s))
  	  (* 1000 (- org-lowest-priority org-default-priority))
  	(* 1000 (- org-lowest-priority
             (string-to-char (match-string 2 s))))))))

Here again, an entry without any priority cookie is treated as default
priority. Note that this function is usually called with the headline
itself as the argument, even without a priority cookie, not a parent
headline containing one.

`org-get-priority-function', although broken due to a missing argument,
gives another hint. Here is its docstring:

  Function to extract the priority from a string.
  The string is normally the headline.  If this is nil Org computes the
  priority from the priority cookie like [#A] in the headline.  It returns
  an integer, increasing by 1000 for each priority level.
  The user can set a different function here, which should take a string
  as an argument and return the numeric priority.

This one also couldn't care less about inheritance.

All in all, I'm not convinced "PRIORITY" inheritance was an intended
feature. At least the current change brings consistency to priority
handling: now there is no inheritance anywhere.

Another option is to make everything use inheritance. However,
`org-get-priority' and `org-get-priority-function' need to be sorted out
(e.g., what string should be passed?). It may be not as easy as it
seems.