Re: tags-todo agenda shoud not ignore DONE items

Re: tags-todo agenda shoud not ignore DONE items

[James N V Cash]

[-- Attachment #1: Type: text/plain, Size: 380 bytes --]


Bastien <bzg@gnu.org> writes:

> Confirming this as an issue, if someone wants to fix it.

I've tried taking a shot at addressing this issue via the patch attached.

As mentioned in the thread, might potentially cause issues by changing how it matches -- should there be an option? Or is the idea just that if you really do want to exclude done tags, you'd query like "/!DONE"?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: matcher allow all tags --]
[-- Type: text/x-diff, Size: 551 bytes --]

diff --git a/lisp/org.el b/lisp/org.el
index 020cfc873..3db1add8b 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -11691,7 +11691,7 @@ See also `org-scan-tags'."
 		       `(and ,tagsmatcher ,todomatcher)
 		     (or tagsmatcher todomatcher t))))
       (when org--matcher-tags-todo-only
-	(setq matcher `(and (member todo org-not-done-keywords) ,matcher)))
+	(setq matcher `(and (member todo org-todo-keywords-1) ,matcher)))
       (cons match0 `(lambda (todo tags-list level) ,matcher)))))
 
 (defun org--tags-expand-group (group tag-groups expanded)

Re: tags-todo agenda shoud not ignore DONE items

[Kyle Meyer]

[ Sorry for the slow reply here, and thank you for looking into this. ]

James N V Cash writes:

> Bastien <bzg@gnu.org> writes:
>
>> Confirming this as an issue, if someone wants to fix it.
>
> I've tried taking a shot at addressing this issue via the patch attached.
>
> As mentioned in the thread, might potentially cause issues by changing
> how it matches -- should there be an option? Or is the idea just that
> if you really do want to exclude done tags, you'd query like "/!DONE"?

Yep, that's the idea, I think.  It's mentioned in both 942b6267a
(org-agenda: `tags-todo' command type includes DONE keywords,
2017-04-18) and 2fb129b5c (`org-scan-tags' retrieve all TODO keywords,
not only not-done ones, 2017-08-17).

>
> diff --git a/lisp/org.el b/lisp/org.el
> index 020cfc873..3db1add8b 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -11691,7 +11691,7 @@ See also `org-scan-tags'."
>  		       `(and ,tagsmatcher ,todomatcher)
>  		     (or tagsmatcher todomatcher t))))
>        (when org--matcher-tags-todo-only
> -	(setq matcher `(and (member todo org-not-done-keywords) ,matcher)))
> +	(setq matcher `(and (member todo org-todo-keywords-1) ,matcher)))
>        (cons match0 `(lambda (todo tags-list level) ,matcher)))))
>  
>  (defun org--tags-expand-group (group tag-groups expanded)

This does resolve the reported issue with the b tags-todo set reported
in <https://orgmode.org/list/m2ftazm3l1.fsf@brautaset.org>.  However, it
breaks the org-tags-view todo-only operation (agenda's M): it no longer
filters out todos in a DONE state.  This behavior is explicitly
specified in the manual:

  Using ‘M-x org-agenda M’ or ‘/!’ does not match TODO keywords in a
  DONE state.

In <https://orgmode.org/list/87d061auiw.fsf@kyleam.com>, I asked,
"shouldn't tags-todo consider only none-DONE todo items, in the same
spirit as m versus M?"  And, looking at this again, it's not only in
spirit; the tags-todo agenda view and org-tags-view's todo-only are
really directly linked:

   ((eq type 'tags-todo)
    (org-let lprops '(org-tags-view '(4) org-match)))

After rereading the thread at m2ftazm3l1.fsf@brautaset.org, I'm still
not convinced we should break that link.

Re: tags-todo agenda shoud not ignore DONE items

[Bastien]

Kyle Meyer <kyle@kyleam.com> writes:

>> As mentioned in the thread, might potentially cause issues by changing
>> how it matches -- should there be an option? Or is the idea just that
>> if you really do want to exclude done tags, you'd query like "/!DONE"?
>
> Yep, that's the idea, I think.

Closing this issue right now -- feel free to report again if if still
needs to be resolved.

tags-todo org-agenda-custom-command weirdness

[Stig Brautaset]

Hi,

I'm seeing some weirdness with tags-todo vs tags custom agenda commands.
Below is the smallest case I've managed to narrow it down to. I would
*expect* that all the a1-4 commands return the same results, and that
the b1-4 return the same results. That is not what I'm seeing, however.

(setq-default org-agenda-custom-commands
	      '(("a1" "A1" tags-todo "-PROJ/TODO")
                ("a2" "A2" tags "-PROJ/TODO")
		("a3" "A3" ((tags-todo "-PROJ/TODO")))
		("a4" "A4" ((tags "-PROJ/TODO")))
		
		("b1" "B1" tags-todo "-PROJ/DONE")
		("b2" "B2" tags "-PROJ/DONE")
		("b3" "B3" ((tags-todo "-PROJ/DONE")))
		("b4" "B4" ((tags "-PROJ/DONE")))))


The effect of a1 & a2 appears identical to eachother, and a3 & a4 too,
but a1-2 and a3-4 differ (in that the former seems to include scheduled
items, but the latter does not.)

What I'm seeing for b1-4 is even weirder. Here I observe b1, b2 and b4
have identical behaviour[*], but b3 is the odd one out: while the other
three return a list of DONE non-project tasks, b3 doesn't find anything.

[*] I suspect the reason b2 and b4 are identical is that I have no
scheduled DONE tasks.

I also see this with =m= vs =M= in the org-agenda menu. I see it even
when starting emacs with -q, so it appears to not be some spectactular
breakage on my part. Is anyone else seing this?


Stig

Re: tags-todo org-agenda-custom-command weirdness

[Kyle Meyer]

Stig Brautaset writes:

> I'm seeing some weirdness with tags-todo vs tags custom agenda commands.
> Below is the smallest case I've managed to narrow it down to. I would
> *expect* that all the a1-4 commands return the same results, and that
> the b1-4 return the same results. That is not what I'm seeing, however.
>
> (setq-default org-agenda-custom-commands
>             '(("a1" "A1" tags-todo "-PROJ/TODO")
>               ("a2" "A2" tags "-PROJ/TODO")
>               ("a3" "A3" ((tags-todo "-PROJ/TODO")))
>               ("a4" "A4" ((tags "-PROJ/TODO")))
>               
>               ("b1" "B1" tags-todo "-PROJ/DONE")
>               ("b2" "B2" tags "-PROJ/DONE")
>               ("b3" "B3" ((tags-todo "-PROJ/DONE")))
>               ("b4" "B4" ((tags "-PROJ/DONE")))))
>

I agree with your expectation that a1-4 should be the same.  Why do you
expect that b1-4 would be?  Shouldn't tags-todo consider only none-DONE
todo items, in the same spirit as m versus M?  In other words, I'd
expect b1 and b3 to be empty by definition.

> The effect of a1 & a2 appears identical to eachother, and a3 & a4 too,
> but a1-2 and a3-4 differ (in that the former seems to include scheduled
> items, but the latter does not.)

It's helpful if you provide a minimal test file.  I've tried to come up
with one that I think should capture what you're describing.

--8<---------------cut here---------------start------------->8---
* TODO h1                                                               :PROJ:
* TODO h2
* DONE h3                                                               :PROJ:
* DONE h4
* TODO h5                                                               :PROJ:
SCHEDULED: <2020-06-14 Sun>
* TODO h6
SCHEDULED: <2020-06-14 Sun>
* DONE h7                                                               :PROJ:
SCHEDULED: <2020-06-14 Sun>
* DONE h8
SCHEDULED: <2020-06-14 Sun>
--8<---------------cut here---------------end--------------->8---

With that, a1-4 all show:

    scratch:    TODO h2
    scratch:    TODO h6

That doesn't match what you're seeing.  I'm testing with 706970 checked
out, the commit you reported in your follow-up message.

> What I'm seeing for b1-4 is even weirder. Here I observe b1, b2 and b4
> have identical behaviour[*], but b3 is the odd one out: while the other
> three return a list of DONE non-project tasks, b3 doesn't find anything.
>
> [*] I suspect the reason b2 and b4 are identical is that I have no
> scheduled DONE tasks.

For b2 and b4, I see

    scratch:    DONE h4
    scratch:    DONE h8

b1 and b3 are empty, which I think is expected given the tags-todo type.

Re: tags-todo org-agenda-custom-command weirdness

[Stig Brautaset]

Kyle Meyer <kyle@kyleam.com> writes:

> Stig Brautaset writes:
>
>> I'm seeing some weirdness with tags-todo vs tags custom agenda commands.
>> Below is the smallest case I've managed to narrow it down to. I would
>> *expect* that all the a1-4 commands return the same results, and that
>> the b1-4 return the same results. That is not what I'm seeing, however.
>>
>> (setq-default org-agenda-custom-commands
>>             '(("a1" "A1" tags-todo "-PROJ/TODO")
>>               ("a2" "A2" tags "-PROJ/TODO")
>>               ("a3" "A3" ((tags-todo "-PROJ/TODO")))
>>               ("a4" "A4" ((tags "-PROJ/TODO")))
>>               
>>               ("b1" "B1" tags-todo "-PROJ/DONE")
>>               ("b2" "B2" tags "-PROJ/DONE")
>>               ("b3" "B3" ((tags-todo "-PROJ/DONE")))
>>               ("b4" "B4" ((tags "-PROJ/DONE")))))
>>
>
> I agree with your expectation that a1-4 should be the same.  Why do you
> expect that b1-4 would be?  Shouldn't tags-todo consider only none-DONE
> todo items, in the same spirit as m versus M?  In other words, I'd
> expect b1 and b3 to be empty by definition.

I expected b1-4 to all yield the same result because I expect
`tags-todo` to search all TODO-items, not just TODO items where the
state is "TODO".


>> The effect of a1 & a2 appears identical to eachother, and a3 & a4 too,
>> but a1-2 and a3-4 differ (in that the former seems to include scheduled
>> items, but the latter does not.)
>
> It's helpful if you provide a minimal test file.  I've tried to come up
> with one that I think should capture what you're describing.

Mea culpa, I should have done this. 

>
> --8<---------------cut here---------------start------------->8---
> * TODO h1                                                               :PROJ:
> * TODO h2
> * DONE h3                                                               :PROJ:
> * DONE h4
> * TODO h5                                                               :PROJ:
> SCHEDULED: <2020-06-14 Sun>
> * TODO h6
> SCHEDULED: <2020-06-14 Sun>
> * DONE h7                                                               :PROJ:
> SCHEDULED: <2020-06-14 Sun>
> * DONE h8
> SCHEDULED: <2020-06-14 Sun>
> --8<---------------cut here---------------end--------------->8---
>
> With that, a1-4 all show:
>
>     scratch:    TODO h2
>     scratch:    TODO h6
>
> That doesn't match what you're seeing.  I'm testing with 706970 checked
> out, the commit you reported in your follow-up message.

For me a2 and a4 returns the same, but a1 and a3 just returns TODO h2.
However, I can reproduce what you're seeing by setting this:

(setq org-agenda-todo-ignore-scheduled nil)


>> What I'm seeing for b1-4 is even weirder. Here I observe b1, b2 and b4
>> have identical behaviour[*], but b3 is the odd one out: while the other
>> three return a list of DONE non-project tasks, b3 doesn't find anything.
>>
>> [*] I suspect the reason b2 and b4 are identical is that I have no
>> scheduled DONE tasks.
>
> For b2 and b4, I see
>
>     scratch:    DONE h4
>     scratch:    DONE h8
>
> b1 and b3 are empty, which I think is expected given the tags-todo type.

Well, I disagree :-)

This behaviour of `tags-todo` seems inconsistent to me. If `todo` can
find DONE items, why shouldn't `tags-todo` do the same?

Stig

Re: tags-todo org-agenda-custom-command weirdness

[Kyle Meyer]

Stig Brautaset writes:

> For me a2 and a4 returns the same, but a1 and a3 just returns TODO h2.
> However, I can reproduce what you're seeing by setting this:
>
> (setq org-agenda-todo-ignore-scheduled nil)

So, provided you've configured org-agenda-todo-ignore-scheduled and
org-agenda-tags-todo-honor-ignore-options away from their default values
of nil, the a1-4 behavior you described is as advertised, correct?

>> b1 and b3 are empty, which I think is expected given the tags-todo type.
>
> Well, I disagree :-)
>
> This behaviour of `tags-todo` seems inconsistent to me. If `todo` can
> find DONE items, why shouldn't `tags-todo` do the same?

Perhaps it should.  The behavior has been that way for a long time, and
the code makes it look very deliberate.  That of course is not an
argument that it should be that way, just one basis for expectations
(and a reason to be wary of breaking workflows).

And it looks like it was actually supposed to change to your preference
in 2017.  There was a report [0] that essentially boils down to what
you're saying, I think.  In response, 942b6267a (org-agenda: `tags-todo'
command type includes DONE keywords, 2017-04-18) was applied, but then
reverted for reasons not related to the intended change in behavior [1].

There was then a follow-up in 2fb129b5c (`org-scan-tags' retrieve all
TODO keywords, not only not-done ones, 2017-08-17).  As far as I can
tell, that was supposed to achieve the behavior you're after but didn't.
I don't have time to dig much at the moment, but quickly stepping
through org-scan-tags, I think the issue is that the MATCHER argument
still filters out done states.


[0] https://orgmode.org/list/CAF96XX0XxhPKjAxy0dQmOiy3rnT+DUok4p1y71F1AwyJaNL-MA@mail.gmail.com/
[1] https://orgmode.org/list/874lt89fi2.fsf@free.fr/

tags-todo agenda shoud not ignore DONE items (was: tags-todo org-agenda-custom-command weirdness)

[Bastien]

Kyle Meyer <kyle@kyleam.com> writes:

>> This behaviour of `tags-todo` seems inconsistent to me. If `todo` can
>> find DONE items, why shouldn't `tags-todo` do the same?
>
> Perhaps it should.  The behavior has been that way for a long time, and
> the code makes it look very deliberate.  That of course is not an
> argument that it should be that way, just one basis for expectations
> (and a reason to be wary of breaking workflows).
>
> And it looks like it was actually supposed to change to your preference
> in 2017.  There was a report [0] that essentially boils down to what
> you're saying, I think.  In response, 942b6267a (org-agenda: `tags-todo'
> command type includes DONE keywords, 2017-04-18) was applied, but then
> reverted for reasons not related to the intended change in behavior [1].
>
> There was then a follow-up in 2fb129b5c (`org-scan-tags' retrieve all
> TODO keywords, not only not-done ones, 2017-08-17).  As far as I can
> tell, that was supposed to achieve the behavior you're after but didn't.
> I don't have time to dig much at the moment, but quickly stepping
> through org-scan-tags, I think the issue is that the MATCHER argument
> still filters out done states.
>
>
> [0] https://orgmode.org/list/CAF96XX0XxhPKjAxy0dQmOiy3rnT+DUok4p1y71F1AwyJaNL-MA@mail.gmail.com/
> [1] https://orgmode.org/list/874lt89fi2.fsf@free.fr/

Confirming this as an issue, if someone wants to fix it.

-- 
 Bastien

Re: tags-todo agenda shoud not ignore DONE items

[Bastien]

Bastien <bzg@gnu.org> writes:

> Confirming this as an issue, if someone wants to fix it.

This should be fixed now with 823f9744e in maint, tags-todo should now
include DONE headings.

-- 
 Bastien

Re: tags-todo agenda shoud not ignore DONE items

[Kyle Meyer]

Bastien writes:

> Bastien <bzg@gnu.org> writes:
>
>> Confirming this as an issue, if someone wants to fix it.
>
> This should be fixed now with 823f9744e in maint, tags-todo should now
> include DONE headings.

823f9744e looks like a regression because it removes the distinction
between `tags' and `tags-todo'.  Consider the following file

--8<---------------cut here---------------start------------->8---
* h1                                                                   :atag:
* TODO h2                                                              :atag:
* DONE h3                                                              :atag:
* h4
* TODO h5
* DONE h6
--8<---------------cut here---------------end--------------->8---

and the following configuration

  (setq org-agenda-custom-commands
        '(("1" tags "atag")
          ("2" tags-todo "atag")))

Before the above commit, 1 should show

  scratch:    h1                                                                          :atag:
  scratch:    TODO h2                                                                     :atag:
  scratch:    DONE h3                                                                     :atag:

and 2

  scratch:    TODO h2                                                                     :atag:

That matches my expectations, though the request in this thread is that
2 includes "DONE h3" as well.

With 823f9744e, both 1 and 2 show

  scratch:    h1                                                                          :atag:
  scratch:    TODO h2                                                                     :atag:
  scratch:    DONE h3                                                                     :atag:

Note the inclusion of a non-TODO entry for 2 (tags-todo).

---

James Cash sent a followup patch to this in a detached thread:

  https://orgmode.org/list/87tuvyaopv.fsf@gmail.com

As I mentioned in that thread (<87361d7p3q.fsf@kyleam.com>) and
suggested in this thread (<87d061auiw.fsf@kyleam.com>), I think tags and
tags-todo should stay aligned with agenda's m/M, with tags-todo
excluding DONE items just as M does.

Re: tags-todo agenda shoud not ignore DONE items

[Bastien]

Kyle Meyer <kyle@kyleam.com> writes:

> 823f9744e looks like a regression because it removes the distinction
> between `tags' and `tags-todo'.

I reverted this commit.

> James Cash sent a followup patch to this in a detached thread:
>
>   https://orgmode.org/list/87tuvyaopv.fsf@gmail.com
>
> As I mentioned in that thread (<87361d7p3q.fsf@kyleam.com>) and
> suggested in this thread (<87d061auiw.fsf@kyleam.com>), I think tags and
> tags-todo should stay aligned with agenda's m/M, with tags-todo
> excluding DONE items just as M does.

Sorry I missed these replies, I see now.  If you think there are
places in the documentation that needs some fixing or clarification
please go ahead as you see fit.

Thanks,

-- 
 Bastien