Highlighting bug with org-agenda-set-restriction-lock

Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Hi,

The highlighting has changed for this functionality.  I finally updated
to master (1200+ commits) yesterday and when I use C-c C-x < the entire
subtree is now highlighted in bright yellow on my Windows Emacs.

GNU Emacs 23.3.1 (i386-mingw-nt5.1.2600) of 2011-03-10 on 3249CTO

Org-mode version 7.9.2 (release_7.9.2-2030-g29a75d @
d:/bin/org-mode/lisp/)

This used to highlight only the first heading of the subtree.
Highlighting the entire subtree is _very_ distracting and makes working
in the org file on the narrowed subtree difficult (for me).

I can fix this by changing the face and completely removing the
highlight but I'd prefer the old functionality if that is possible.

I also noticed that the main heading is now also highlighted in the
agenda which is okay I guess but also different from how this used to
work :)

I haven't tried this on  my Linux system at home yet.  Will do that this
weekend.

Thanks,
Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Bernt Hansen <bernt@norang.ca> writes:

> I also noticed that the main heading is now also highlighted in the
> agenda which is okay I guess but also different from how this used to
> work :)

This is unrelated - it's highlighted because it is my current clocking
task. :)

Thanks,
Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Carsten Dominik]

On 5 apr. 2013, at 15:12, Bernt Hansen <bernt@norang.ca> wrote:

> Hi,
> 
> The highlighting has changed for this functionality.  I finally updated
> to master (1200+ commits) yesterday and when I use C-c C-x < the entire
> subtree is now highlighted in bright yellow on my Windows Emacs.
> 
> GNU Emacs 23.3.1 (i386-mingw-nt5.1.2600) of 2011-03-10 on 3249CTO
> 
> Org-mode version 7.9.2 (release_7.9.2-2030-g29a75d @
> d:/bin/org-mode/lisp/)
> 
> This used to highlight only the first heading of the subtree.
> Highlighting the entire subtree is _very_ distracting and makes working
> in the org file on the narrowed subtree difficult (for me).
> 
> I can fix this by changing the face and completely removing the
> highlight but I'd prefer the old functionality if that is possible.

This is causes by this commit:

commit aa0e0068de109eef2ac7897c4659d545b351de01
Author: Bastien Guerry <bzg@altern.org>
Date:   Sat Feb 16 23:09:57 2013 +0100

    org-agenda.el (org-agenda-set-restriction-lock): Put the overlay until the e
    
    * org-agenda.el (org-agenda-set-restriction-lock): Put the
    overlay until the end of the subtree, not the end of the
    headline.
    
    When the agenda restriction is on, user expect agenda views to check
    every entry in the subtree.  If a user add an entry outside of the
    overlay without noticing it, this entry will not be checked and the
    user will wonder why.  Put the end of the overlay at the end of the
    subtree so that the user always knows if the entries she is adding
    are within the current restriction.
    
    We might need to find a less instrusive overlay color, though.


So Bastien felt that the entire subtree should have an overlay, but
maybe a softer color.  I would also prefer the highlight just to be
on the headline as it used to be.

Bastien, how do you feel about reverting this change?  I see why
you did it, but it is not so practical after all if you want to
work in this mode for extended time.

- Carsten


> 
> I also noticed that the main heading is now also highlighted in the
> agenda which is okay I guess but also different from how this used to
> work :)
> 
> I haven't tried this on  my Linux system at home yet.  Will do that this
> weekend.
> 
> Thanks,
> Bernt
> 

Re: Highlighting bug with org-agenda-set-restriction-lock

[Carsten Dominik]

On 5 apr. 2013, at 16:12, Carsten Dominik <carsten.dominik@gmail.com> wrote:

> 
> On 5 apr. 2013, at 15:12, Bernt Hansen <bernt@norang.ca> wrote:
> 
>> Hi,
>> 
>> The highlighting has changed for this functionality.  I finally updated
>> to master (1200+ commits) yesterday and when I use C-c C-x < the entire
>> subtree is now highlighted in bright yellow on my Windows Emacs.
>> 
>> GNU Emacs 23.3.1 (i386-mingw-nt5.1.2600) of 2011-03-10 on 3249CTO
>> 
>> Org-mode version 7.9.2 (release_7.9.2-2030-g29a75d @
>> d:/bin/org-mode/lisp/)
>> 
>> This used to highlight only the first heading of the subtree.
>> Highlighting the entire subtree is _very_ distracting and makes working
>> in the org file on the narrowed subtree difficult (for me).
>> 
>> I can fix this by changing the face and completely removing the
>> highlight but I'd prefer the old functionality if that is possible.
> 
> This is causes by this commit:
> 
> commit aa0e0068de109eef2ac7897c4659d545b351de01
> Author: Bastien Guerry <bzg@altern.org>
> Date:   Sat Feb 16 23:09:57 2013 +0100
> 
>    org-agenda.el (org-agenda-set-restriction-lock): Put the overlay until the e
> 
>    * org-agenda.el (org-agenda-set-restriction-lock): Put the
>    overlay until the end of the subtree, not the end of the
>    headline.
> 
>    When the agenda restriction is on, user expect agenda views to check
>    every entry in the subtree.  If a user add an entry outside of the
>    overlay without noticing it, this entry will not be checked and the
>    user will wonder why.  Put the end of the overlay at the end of the
>    subtree so that the user always knows if the entries she is adding
>    are within the current restriction.
> 
>    We might need to find a less instrusive overlay color, though.
> 
> 
> So Bastien felt that the entire subtree should have an overlay, but
> maybe a softer color.  I would also prefer the highlight just to be
> on the headline as it used to be.
> 
> Bastien, how do you feel about reverting this change?  I see why
> you did it, but it is not so practical after all if you want to
> work in this mode for extended time.


I now also see that the patch did not achieve what the purpose of it was.  The problem is not the end of the overlay, but the position of the org-agenda-restriction-end marker.  Indeed, if you add tasks after this marker, they will not be included in the search.  There is not good work-around for this I can think of, so it is a problem we will have to live with.

- Carsten

Re: Highlighting bug with org-agenda-set-restriction-lock

[Carsten Dominik]

On 5 apr. 2013, at 16:23, Carsten Dominik <carsten.dominik@gmail.com> wrote:

> 
> On 5 apr. 2013, at 16:12, Carsten Dominik <carsten.dominik@gmail.com> wrote:
> 
>> 
>> On 5 apr. 2013, at 15:12, Bernt Hansen <bernt@norang.ca> wrote:
>> 
>>> Hi,
>>> 
>>> The highlighting has changed for this functionality.  I finally updated
>>> to master (1200+ commits) yesterday and when I use C-c C-x < the entire
>>> subtree is now highlighted in bright yellow on my Windows Emacs.
>>> 
>>> GNU Emacs 23.3.1 (i386-mingw-nt5.1.2600) of 2011-03-10 on 3249CTO
>>> 
>>> Org-mode version 7.9.2 (release_7.9.2-2030-g29a75d @
>>> d:/bin/org-mode/lisp/)
>>> 
>>> This used to highlight only the first heading of the subtree.
>>> Highlighting the entire subtree is _very_ distracting and makes working
>>> in the org file on the narrowed subtree difficult (for me).
>>> 
>>> I can fix this by changing the face and completely removing the
>>> highlight but I'd prefer the old functionality if that is possible.
>> 
>> This is causes by this commit:
>> 
>> commit aa0e0068de109eef2ac7897c4659d545b351de01
>> Author: Bastien Guerry <bzg@altern.org>
>> Date:   Sat Feb 16 23:09:57 2013 +0100
>> 
>>   org-agenda.el (org-agenda-set-restriction-lock): Put the overlay until the e
>> 
>>   * org-agenda.el (org-agenda-set-restriction-lock): Put the
>>   overlay until the end of the subtree, not the end of the
>>   headline.
>> 
>>   When the agenda restriction is on, user expect agenda views to check
>>   every entry in the subtree.  If a user add an entry outside of the
>>   overlay without noticing it, this entry will not be checked and the
>>   user will wonder why.  Put the end of the overlay at the end of the
>>   subtree so that the user always knows if the entries she is adding
>>   are within the current restriction.
>> 
>>   We might need to find a less instrusive overlay color, though.
>> 
>> 
>> So Bastien felt that the entire subtree should have an overlay, but
>> maybe a softer color.  I would also prefer the highlight just to be
>> on the headline as it used to be.
>> 
>> Bastien, how do you feel about reverting this change?  I see why
>> you did it, but it is not so practical after all if you want to
>> work in this mode for extended time.
> 
> 
> I now also see that the patch did not achieve what the purpose of it was.  The problem is not the end of the overlay, but the position of the org-agenda-restriction-end marker.  Indeed, if you add tasks after this marker, they will not be included in the search.  There is not good work-around for this I can think of, so it is a problem we will have to live with.


I am fixing this - please let me know if I should revert my fix...

Bernt, should be working in master.

- Carsten

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> I now also see that the patch did not achieve what the purpose of it
> was.  The problem is not the end of the overlay, but the position of
> the org-agenda-restriction-end marker.  Indeed, if you add tasks after
> this marker, they will not be included in the search.  There is not
> good work-around for this I can think of, so it is a problem we will
> have to live with.
>
> - Carsten

I would prefer the original highlighting behaviour as well.
I normally narrow the buffer to the subtree as well when working on it
-- and the entire narrowed buffer is bright gaudy yellow currently :).

My workaround for the adding items to the ends is to remark the subtree
restriction.  I have added speed keys P, F, W, and N in my setup for
narrow to project, narrow to file, widen, and narrow to subtree
respectively.

I normally narrow to project with P (on any task in the project narrows
the current buffer to the project.  In the agenda P sets the restriction
lock on the project -- so if I've added anything new at the end of the
project I can just hit P on any task in the agenda and the restriction
lock is reset appropriately including all of the (new) tasks in the
project.  (My definition of a project is any todo keyword task that has
at least one subtask with a todo keyword.)

Regards,
Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Carsten Dominik]

On 5 apr. 2013, at 16:46, Bernt Hansen <bernt@norang.ca> wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> I now also see that the patch did not achieve what the purpose of it
>> was.  The problem is not the end of the overlay, but the position of
>> the org-agenda-restriction-end marker.  Indeed, if you add tasks after
>> this marker, they will not be included in the search.  There is not
>> good work-around for this I can think of, so it is a problem we will
>> have to live with.
>> 
>> - Carsten
> 
> I would prefer the original highlighting behaviour as well.
> I normally narrow the buffer to the subtree as well when working on it
> -- and the entire narrowed buffer is bright gaudy yellow currently :).
> 
> My workaround for the adding items to the ends is to remark the subtree
> restriction.  I have added speed keys P, F, W, and N in my setup for
> narrow to project, narrow to file, widen, and narrow to subtree
> respectively.
> 
> I normally narrow to project with P (on any task in the project narrows
> the current buffer to the project.  In the agenda P sets the restriction
> lock on the project -- so if I've added anything new at the end of the
> project I can just hit P on any task in the agenda and the restriction
> lock is reset appropriately including all of the (new) tasks in the
> project.  (My definition of a project is any todo keyword task that has
> at least one subtask with a todo keyword.)

Well, that kind of a work-around, yes.  I meant something like telling
the marker to insert any new text in front of it, but there is now
good way.

I like the speed keys!

- Carsten

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> I am fixing this - please let me know if I should revert my fix...
>
> Bernt, should be working in master.

Works much better!  Thanks!!

Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Carsten Dominik]

On 5.4.2013, at 17:48, Bernt Hansen <bernt@norang.ca> wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> I am fixing this - please let me know if I should revert my fix...
>> 
>> Bernt, should be working in master.
> 
> Works much better!  Thanks!!

Hi Bernt,

Bastien has introduced a variable for this, you need to set it to get highlight in the headline only:

http://orgmode.org/cgit.cgi/org-mode.git/commit/?id=b37e27

(setq org-agenda-restriction-lock-highlight-subtree nil)

The face is now less intrusive, and you can change it of course.

- Carsten

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bastien]

Hi Bernt,

Bernt Hansen <bernt@norang.ca> writes:

> My workaround for the adding items to the ends is to remark the subtree
> restriction.  I have added speed keys P, F, W, and N in my setup for
> narrow to project, narrow to file, widen, and narrow to subtree
> respectively.

Note that you can narrow to subtree with the `s' speed key already.

2 cts -- and thanks for upgrading!  Your feedback is most welcome
and awaited :)

-- 
 Bastien

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Bastien <bzg@gnu.org> writes:

> Hi Bernt,
>
> Bernt Hansen <bernt@norang.ca> writes:
>
>> My workaround for the adding items to the ends is to remark the subtree
>> restriction.  I have added speed keys P, F, W, and N in my setup for
>> narrow to project, narrow to file, widen, and narrow to subtree
>> respectively.
>
> Note that you can narrow to subtree with the `s' speed key already.
>
> 2 cts -- and thanks for upgrading!  Your feedback is most welcome
> and awaited :)

I've already remapped 's' to save all org buffers and I want the same
key in both the agenda and org file so N works for me :)

--------------------------------------------------------------------------------

I've noticed some differences since upgrading for creating headlines but
I haven't checked if this is due to my local setup yet.

Here is what I'm currently experiencing:

I used to be able to just have a bunch of lines likes this

,----
| one
| two
| three
| four
`----

and M-S-RET in the first column would create a TODO headline with this
as the heading

ie.

,----
| * TODO one
| two
| three
| four
`----

but now it seems to jump past the line before creating the heading.

,----
| one
| 
| * TODO
| two
| three
| four
`----

I have a heading hook that runs and inserts a timestamp when headings
are created so using this as a workaround and converting this to a list
with C-u C-c - and then to headings with C-c * doesn't give me the
result I want (inactive timestamps on all headings that were created)

I haven't tried this with a minimal emacs setup yet.

Regards,
Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bastien]

Hi Bernt,

Bernt Hansen <bernt@norang.ca> writes:

> I've noticed some differences since upgrading for creating headlines but
> I haven't checked if this is due to my local setup yet.

Feel free to throw all oddities that you notice, even with no deep
test, it will help a lot.

> Here is what I'm currently experiencing:
>
> I used to be able to just have a bunch of lines likes this
>
> ,----
> | one
> | two
> | three
> | four
> `----
>
> and M-S-RET in the first column would create a TODO headline with this
> as the heading
>
> ie.
>
> ,----
> | * TODO one
> | two
> | three
> | four
> `----
>
> but now it seems to jump past the line before creating the heading.
>
> ,----
> | one
> | 
> | * TODO
> | two
> | three
> | four
> `----

I get the correct behavior with emacs -Q and latest master, so it must
surely be in your setup.

> I have a heading hook that runs and inserts a timestamp when headings
> are created so using this as a workaround and converting this to a list
> with C-u C-c - and then to headings with C-c * doesn't give me the
> result I want (inactive timestamps on all headings that were created)

You can directly convert a normal line to a heading with C-c *

Also, C-u C-c - will have the same effect if no region is selected: 
to convert the normal line at point to a list item.

Beware that the default behavior changed when you select a region
and want to convert it to a list: now C-c - will convert all lines
in the region to list items; use C-u C-c - to convert the whole
region as a single item.  This behavior is more in sync with the 
one of C-c *.

HTH,

-- 
 Bastien

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Bastien <bzg@gnu.org> writes:

> Bernt Hansen <bernt@norang.ca> writes:
>
>> I've noticed some differences since upgrading for creating headlines but
>> I haven't checked if this is due to my local setup yet.
>
> Feel free to throw all oddities that you notice, even with no deep
> test, it will help a lot.
>
>> Here is what I'm currently experiencing:
>>
>> I used to be able to just have a bunch of lines likes this
>>
>> ,----
>> | one
>> | two
>> | three
>> | four
>> `----
>>
>> and M-S-RET in the first column would create a TODO headline with this
>> as the heading
>>
>> ie.
>>
>> ,----
>> | * TODO one
>> | two
>> | three
>> | four
>> `----
>>
>> but now it seems to jump past the line before creating the heading.
>>
>> ,----
>> | one
>> | 
>> | * TODO
>> | two
>> | three
>> | four
>> `----
>
> I get the correct behavior with emacs -Q and latest master, so it must
> surely be in your setup.
>
>> I have a heading hook that runs and inserts a timestamp when headings
>> are created so using this as a workaround and converting this to a list
>> with C-u C-c - and then to headings with C-c * doesn't give me the
>> result I want (inactive timestamps on all headings that were created)

Hi Bastien,

I'm not convinced it is in my setup yet.

emacs -Q works for me ... but that is org 7.5 in my
GNU Emacs 23.2.1 (i486-pc-linux-gnu, GTK+ Version 2.20.1) of 2013-01-08
on murphy, modified by Debian

My minimal emacs setup has the same broken behaviour

,----[ minimal.emacs ]
| (add-to-list 'load-path "~/git/org-mode/lisp/")
| ;(add-to-list 'load-path (expand-file-name "~/git/org-mode/lisp"))
| (add-to-list 'auto-mode-alist '("\\.\\(org\\|org_archive\\|txt\\)$" . org-mode))
| (require 'org)
| 
| (global-set-key "\C-cl" 'org-store-link)
| (global-set-key "\C-ca" 'org-agenda)
| (global-set-key "\C-cb" 'org-iswitchb)
`----

,----[ minimal-emacs ]
| #!/bin/sh
| TESTEL=
| TESTFILE=/tmp/test.el
| if test -e $TESTFILE
| then
|   TESTEL="-l /tmp/test.el"
| fi
| emacs -q -l ~/minimal.emacs $TESTEL $1
`----

/tmp/test.el does not exist currently on my system

I can reproduce this with 

~/bin/minimal-emacs

C-x b foo RET
M-x org-mode

Org file has this

,----
| * One
| two
| three
| four
| five
`----

Put the cursor on first column of line two and hit M-S-RET gives me this

,----
| * One
| two
| * TODO 
| three
| four
| five
`----

but I'm expecting 

* TODO two

org-insert-heading-respect-content is nil.

It seems to work correctly when there is no heading above the text
(ie. One is not a headline)

M-x org-version reports
Org-mode version 8.0-pre (release_8.0-pre-299-g40f390 @
/home/bernt/git/org-mode/lisp/)

Regards,
Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bastien]

Hi Bernt,

Bernt Hansen <bernt@norang.ca> writes:

> Org file has this
>
> ,----
> | * One
> | two
> | three
> | four
> | five
> `----
>
> Put the cursor on first column of line two and hit M-S-RET gives me this
>
> ,----
> | * One
> | two
> | * TODO 
> | three
> | four
> | five
> `----

Okay, I see this too.

> It seems to work correctly when there is no heading above the text
> (ie. One is not a headline)

That's what I tested after your previous email, yes.

I need to think harder about this and to get more feedback, because
the new behavior seems the correct one to me.  Also, it is in sync
with the docstring of `org-insert-heading-respect-content':

  Non-nil means insert new headings after the current subtree.  When
  nil, the new heading is created directly after the current line.
                                           ^^^^^

This is about *inserting* a new heading, not about turning the current
line into a heading (which is fine to have when there is no heading
above, of course.)  Also, reverting to the old behavior would move
S-M-RET closer to C-c *.

Let's see if other users find the current behavior annoying, and let
me know how annoying it is to you!

Thanks,

-- 
 Bastien

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Bastien <bzg@gnu.org> writes:

> I need to think harder about this and to get more feedback, because
> the new behavior seems the correct one to me.  Also, it is in sync
> with the docstring of `org-insert-heading-respect-content':
>
>   Non-nil means insert new headings after the current subtree.  When
>   nil, the new heading is created directly after the current line.
>                                            ^^^^^
>
> This is about *inserting* a new heading, not about turning the current
> line into a heading (which is fine to have when there is no heading
> above, of course.)  Also, reverting to the old behavior would move
> S-M-RET closer to C-c *.
>
> Let's see if other users find the current behavior annoying, and let
> me know how annoying it is to you!

OK.  If this is working as designed I'm fine with that.  I'll just
change my habits :)  I actually didn't read the docstring was just
basing it on previous behaviour (which may have been incorrect).

Thanks,
Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Christian Moe]

Didn't `M-RET' on an ordinary line of text also turn that line into a
heading, about the same as doing `C-c *' on that line? At least with
point in the first column (not sure about other cases)? My memory may be
faulty, but lately I've found myself trying to do that, and scratching
my head that it's not working as I thought it did. (I agree, of course,
that the current behavior agrees with the docs, and that the behavior I
seem to remember was redundant, but I also seem to remember it as pretty
handy.)

Yours,
Christian


Bastien writes:

> Hi Bernt,
>
> Bernt Hansen <bernt@norang.ca> writes:
>
>> Org file has this
>>
>> ,----
>> | * One
>> | two
>> | three
>> | four
>> | five
>> `----
>>
>> Put the cursor on first column of line two and hit M-S-RET gives me this
>>
>> ,----
>> | * One
>> | two
>> | * TODO 
>> | three
>> | four
>> | five
>> `----
>
> Okay, I see this too.
>
>> It seems to work correctly when there is no heading above the text
>> (ie. One is not a headline)
>
> That's what I tested after your previous email, yes.
>
> I need to think harder about this and to get more feedback, because
> the new behavior seems the correct one to me.  Also, it is in sync
> with the docstring of `org-insert-heading-respect-content':
>
>   Non-nil means insert new headings after the current subtree.  When
>   nil, the new heading is created directly after the current line.
>                                            ^^^^^
>
> This is about *inserting* a new heading, not about turning the current
> line into a heading (which is fine to have when there is no heading
> above, of course.)  Also, reverting to the old behavior would move
> S-M-RET closer to C-c *.
>
> Let's see if other users find the current behavior annoying, and let
> me know how annoying it is to you!
>
> Thanks,

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bernt Hansen]

Christian Moe <mail@christianmoe.com> writes:

> Didn't `M-RET' on an ordinary line of text also turn that line into a
> heading, about the same as doing `C-c *' on that line? At least with
> point in the first column (not sure about other cases)? My memory may be
> faulty, but lately I've found myself trying to do that, and scratching
> my head that it's not working as I thought it did. (I agree, of course,
> that the current behavior agrees with the docs, and that the behavior I
> seem to remember was redundant, but I also seem to remember it as pretty
> handy.)

Yes it did in previous versions -- with the added change that the
heading hook is run when M-RET (which for me inserts a inactive
timestamp below the new heading).  I don't think the hook is run when
converting headings with C-c *

I definitely found the previous behaviour convenient.

Regards,
Bernt

Re: Highlighting bug with org-agenda-set-restriction-lock

[Bastien]

Hi Bernt,

Bernt Hansen <bernt@norang.ca> writes:

> I definitely found the previous behaviour convenient.

This should be fixed now.

Let me know if it works as expected.  Thanks,

-- 
 Bastien