Manual Ordering and Dynamic Priority

Manual Ordering and Dynamic Priority

[Eduardo Suarez]

I have lots of tasks (todos) and I would like to create a long backlog based on
my perceived priority.

I was thinking to deal with them in the following way:

- divide them in groups (categories or similar),
- manually sort priority for every group,
- mergesort groups, that is, start merging groups in pairs, and manually sort
  for every step the union group until I have a large sorted backlog.

For this to be practical, I would need an easy way to sort manually a group of
tasks and get them assigned automatically a priority (or any other hack) so
that priority ordering matches manual ordering.

Any idea about how to get this done?

If I had to implement it (I don't know lisp), I would assign a property (say
BACKLOG_PRIORITY) for every new task, with value the higher value of any other
tasks in agenda plus ten (for instance). Then I would query a subset of tasks
and sort them manually, swapping their values every time I swap their order. I
would also allow to assign a value directly based on free slots, not to bubble
the whole list for a low priority task.

Does it sound over-engineered? Any idea?

Re: Manual Ordering and Dynamic Priority

[indieterminacy]

Hello Eduardo,

On 31-08-2022 18:13, Eduardo Suarez wrote:
> I have lots of tasks (todos) and I would like to create a long backlog 
> based on
> my perceived priority.
> 
> I was thinking to deal with them in the following way:
> 
> - divide them in groups (categories or similar),
> - manually sort priority for every group,
> - mergesort groups, that is, start merging groups in pairs, and 
> manually sort
>   for every step the union group until I have a large sorted backlog.
> 
> For this to be practical, I would need an easy way to sort manually a 
> group of
> tasks and get them assigned automatically a priority (or any other 
> hack) so
> that priority ordering matches manual ordering.
> 
> Any idea about how to get this done?
> 

While these are non orgmode solutions inside Emacs I hope they can give 
you some ideas of what you want:

https://github.com/sp1ff/elfeed-score
https://www.unwoundstack.com/blog/scoring-elfeed-entries.html

https://www.gnu.org/software/emacs/manual/html_node/gnus/Scoring.html


> If I had to implement it (I don't know lisp), I would assign a property 
> (say
> BACKLOG_PRIORITY) for every new task, with value the higher value of 
> any other
> tasks in agenda plus ten (for instance). Then I would query a subset of 
> tasks
> and sort them manually, swapping their values every time I swap their 
> order. I
> would also allow to assign a value directly based on free slots, not to 
> bubble
> the whole list for a low priority task.
> 
> Does it sound over-engineered? Any idea?

HTH

-- 
Jonathan McHugh
indieterminacy@libre.brussels

Re: Manual Ordering and Dynamic Priority

[Tim Cross]

Eduardo Suarez <esuarez@itccanarias.org> writes:

> I have lots of tasks (todos) and I would like to create a long backlog based on
> my perceived priority.
>
> I was thinking to deal with them in the following way:
>
> - divide them in groups (categories or similar),
> - manually sort priority for every group,
> - mergesort groups, that is, start merging groups in pairs, and manually sort
>   for every step the union group until I have a large sorted backlog.
>
> For this to be practical, I would need an easy way to sort manually a group of
> tasks and get them assigned automatically a priority (or any other hack) so
> that priority ordering matches manual ordering.
>
> Any idea about how to get this done?
>
> If I had to implement it (I don't know lisp), I would assign a property (say
> BACKLOG_PRIORITY) for every new task, with value the higher value of any other
> tasks in agenda plus ten (for instance). Then I would query a subset of tasks
> and sort them manually, swapping their values every time I swap their order. I
> would also allow to assign a value directly based on free slots, not to bubble
> the whole list for a low priority task.
>
> Does it sound over-engineered? Any idea?

It does sound a bit like over-engineering to me :-)

My first question would be "How long have you been using org-mode to
manage your tasks?"

My observation from being a long-term org-mode user (and speaking from
personal experience), is that the first big mistake people tend to make
is to over engineer their setup. I suspect many people, after first
using org-mode, they get somewhat carried away with the
possibilities. They immediately get caught up in this dream of having an
organised world where all their tasks are captured, neat, organised,
prioritised and chaos is under control.

They start by defining a precise and well laid out plan for capturing
tasks and managing them. Tasks are given priorities, effort estimates,
there are capture templates for everything you can possibly think of and
no matter what they are doing, they can open their agenda and
immediately see what to do next, little thought or effort required.

This approach suffers from two fundamental flaws

- Life is by nature chaotic. You cannot eliminate the chaos. The best
  you can do is find ways to make dealing with the chaos less painful.

- Life is about getting things done, not planning to get things
  done. Overly complex management of your tasks is very likely to result
  in more time spent managing the tasks than actually doing the tasks.

My first recommendation for most people is use org with default settings
for the first few months. The defaults are not a bad starting point and
after a few months, you will know what the pain points are for your
specific situation and can then start looking at how to address them.

Be very careful about priorities. The problem is, priorities actually
change faster than you might expect. I actually rarely use priorities as
I find most of the time, they are not adding real benefit. I know what
the priorities are. Only in extremely large or complex projects,
particularly ones with complex dependencies, have I found priorities
useful.

The other point to note is that because you now have an efficient and
quick way to gather task information, you will likely end up with a
growing list of tasks in your backlog. The reality is, a far larger
number of these tasks will never see the light of day than you may
think. I find that very 6 months or so, I go through my backlog of tasks
and move any which have not seen any progress for 12 months into a
backlog archive. These tend to be ideas or tasks which in all honestly
are never going to bubble up to the point of being acted on. This could
be because priorities or requirements have changed, the ideas underlying
the tasks needs more thought or is just a bad idea, the need for the
task no longer relevant etc. Any time spent on prioritising, classifying
or ordering these tasks is largely wasted effort. 

The other problem with priorities are they tend to only represent the
urgency of a task and not the importance of that task. The danger is, if
you only focus on urgency, you become reactive and less proactive. All
your time is spent responding to urgency with little time allocated to
implementing less urgent but important tasks which ironically, will
often result in improvements and less urgent tasks. This can be avoided
if you are disciplined and ensure you also give high priority to
non-urgent, but important tasks. However, this can be hard to do,
especially in a busy environment with lots of conflicting tasks.

Tags can be a very useful mechanism for classifying tasks. However, the
mistake people often make with tags is to create far too many different
tags. You need to be disciplined with tags and keep them to a
minimum. If you have too many, you will spend too much time trying to
decide which tag to apply, will get inconsistency in your tagging and
when it comes time to use the tags, you won't know which tag is
best. Keep tags to as small a number as possible.

For the first 6 - 12 months, my recommendation is to keep things as
simple as possible. Your aim is to minimise the time you need to spend
in planning and organising your tasks while also minimising the tasks
which get missed, forgotten or don't get done in time. Your workflow
needs to be making your life easier, not harder by imposing more things
you have to do. Your looking for ways to help manage the chaos, not
eliminate it and it needs to work in a chaotic environment with
conflicting demands for your time and attention. After 6 - 12 months,
review and identify the 5 most frustrating/time consuming/failing
aspects of your org based workflow and see how they might be addressed
using existing org mode facilities. Implement a solution and use it for
6 months before going through another review and refine process. You
will be surprised how far you can get just using built-in org mode
facilities. At some point, you will likely get to the point where some
real org customisation is needed. However, you will also likely find
very similar customisation has already been done by others whch you will
be able to adapt. 

The key focus should always be in making org mode work for you and not
the other way around. If you find your spending more time managing your
tasks with org mode, your likely got it wrong. It has to make your life
easier or there simply isn't any point to it regardless of how cool,
clever or useful it may seem in theory.

Re: Manual Ordering and Dynamic Priority

[Rudolf Adamkovič]

Tim Cross <theophilusx@gmail.com> writes:

> - Life is about getting things done, not planning to get things
>   done. Overly complex management of your tasks is very likely to
>   result in more time spent managing the tasks than actually doing
>   the tasks.

Let me share a slightly different experience.

I used to minimize the time I spent on planning, wanting to start
"actually doing the tasks".  Looking back, it often led to pointless
work and wasted time.  So, I changed my ways!

These days, I do not mind spending half a day, or more, on planning my
learning week at school.  I also do not mind "more time spent managing
the tasks than actually doing the tasks" because I noticed that it
might, unexpectedly, change the entire equation.  For example, at work,
I noticed that careful planning, which includes design, can save ten
times the time it takes, though one never knows for sure beforehand.

For another example, I spent two days designing my Org Agenda.  I first
made some drawings on paper, thinking carefully about what I want from
it, and then configured Emacs to compute it that way.  It might sound
crazy, for I could have spent those two days "actually doing the tasks",
but I thank myself every day for doing it!

Rudy
-- 
"Strange as it may sound, the power of mathematics rests on its evasion
of all unnecessary thought and on its wonderful saving of mental
operations."
-- Ernst Mach, 1838-1916

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: Manual Ordering and Dynamic Priority

[Tim Cross]

Rudolf Adamkovič <salutis@me.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> - Life is about getting things done, not planning to get things
>>   done. Overly complex management of your tasks is very likely to
>>   result in more time spent managing the tasks than actually doing
>>   the tasks.
>
> Let me share a slightly different experience.
>
> I used to minimize the time I spent on planning, wanting to start
> "actually doing the tasks".  Looking back, it often led to pointless
> work and wasted time.  So, I changed my ways!
>
> These days, I do not mind spending half a day, or more, on planning my
> learning week at school.  I also do not mind "more time spent managing
> the tasks than actually doing the tasks" because I noticed that it
> might, unexpectedly, change the entire equation.  For example, at work,
> I noticed that careful planning, which includes design, can save ten
> times the time it takes, though one never knows for sure beforehand.
>
> For another example, I spent two days designing my Org Agenda.  I first
> made some drawings on paper, thinking carefully about what I want from
> it, and then configured Emacs to compute it that way.  It might sound
> crazy, for I could have spent those two days "actually doing the tasks",
> but I thank myself every day for doing it!
>

The difference here is in what we define as planning. For example, I
would not consider 2 days spend designing how my agenda would work as
task planning - that is design work.

What I'm referring to is having a task management process where you
spend large amounts of time just managing the item 'metadata' - not
actually working on the item at all, but just tracking its state,
progress, priority, etc. 

Planning as a task in itself is very important. I live by the 6Ps
(Piss-poor planning prevents poor performance). However, that planning
effort needs to be focused on moving things forward. Too often, I see
people make the same error with org mode. They develop a complex all
singing and dancing workflow for recording, tracking and managing their
tasks. Before they realise it, they become a slave to the process and
spend lots of time updating task state information, changing priorities
and schedules, making notes which never get read again and tracking
everything at a micro level. AS an extreme example, I saw someone once
who had in their list of tasks "Get out of bed" - honestly, what is
having that task really adding except noise and wasted time. Do you
really need a task telling you to get out of bed? I've seen other
examples where people find they are overwhelmed with the number of tasks
in their task list, but when you look at what they have listed you see
crazy micro level planning e.g.

8:00 - Walk to bus stop
8:15 - Ride bus to Kent st. 
8:45 - Walk from Kent ST. buss stop to office
9:00 - Turn on office computer
9:00 - 11:30 - Do office work
...

Unless you have some sort of memory issue or learning condition,
planning at this level is largely pointless. This may seem extreme, but
I see people do very similar fine grained task logging in projects as
well.

Planning, like effort estimating, is a skill and needs practice. All I
am trying to convey is a warning against being overly ambitious or
having overly high expectations regarding org mode and how/when it can
be beneficial. While you can probably capture everything in your life in
plain text org files, doing so may not actually help make your life
easier. I believe you need to be judicious in what you map out as tasks
and in what data you capture. Too much and there is a danger it all
becomes white noise and you become a slave to the process. Our brains
are amazing things which can handle a lot of complexity easily and at a
fast rate - the interface is still faster and more powerful than org and
we should use it to its maximum and apply org to augment it rather than
replace it. Problem is, someone sees org mode for the first time and
their blown away and suddenly can see how they can generate a
comprehensive, concise and well mapped out process for everything they
do. The pendulum swings to far the other way and suddenly, they now have
an additional time consuming maintenance task which is now using time
previously spent actually getting the job done.

The other danger is of course that we become too dependent on the
technology. We can see this happening now with information like phone
numbers. Even only 20 years ago, most people would memorise and be able
to recall many phone numbers. Recent surveys indicate most people don't
even know the phone number of their spouse! Consider that for a second -
if you lost your mobile phone, how many of the phone numbers of people
close to you can you recall? 

Re: Manual Ordering and Dynamic Priority

[Eduardo Suarez-Santana]

I'd like to show some use cases as examples of long lists that I do not
consider micromanagement.

I started managing tasks with pen and paper, then Palm, Remember the Milk, and
others. None of them worked fine. I reached Org-mode because I have read about
it and it was in my try-list. It has not been an easy way, though.

Indeed, a try-list is for me an use case for org-mode. The fast pace of
technology populates my try-list too fast and I need something to manage it.
Sure, they are all optional tasks, but I don't want to review them often and
I'd like to have a predefined order to deal with them.

Another use case y when dealing with complex systems (servers and services).
There is an overload of issues and improvements to take care of. You can triage
and solve some new issues, but the others need to be captured, documented and
queued.

On Sat, Sep 10, 2022 at 08:05:34AM +1000, Tim Cross wrote:
> 
> Rudolf Adamkovič <salutis@me.com> writes:
> 
> > Tim Cross <theophilusx@gmail.com> writes:
> >
> >> - Life is about getting things done, not planning to get things
> >>   done. Overly complex management of your tasks is very likely to
> >>   result in more time spent managing the tasks than actually doing
> >>   the tasks.
> >
> > Let me share a slightly different experience.
> >
> > I used to minimize the time I spent on planning, wanting to start
> > "actually doing the tasks".  Looking back, it often led to pointless
> > work and wasted time.  So, I changed my ways!
> >
> > These days, I do not mind spending half a day, or more, on planning my
> > learning week at school.  I also do not mind "more time spent managing
> > the tasks than actually doing the tasks" because I noticed that it
> > might, unexpectedly, change the entire equation.  For example, at work,
> > I noticed that careful planning, which includes design, can save ten
> > times the time it takes, though one never knows for sure beforehand.
> >
> > For another example, I spent two days designing my Org Agenda.  I first
> > made some drawings on paper, thinking carefully about what I want from
> > it, and then configured Emacs to compute it that way.  It might sound
> > crazy, for I could have spent those two days "actually doing the tasks",
> > but I thank myself every day for doing it!
> >
> 
> The difference here is in what we define as planning. For example, I
> would not consider 2 days spend designing how my agenda would work as
> task planning - that is design work.
> 
> What I'm referring to is having a task management process where you
> spend large amounts of time just managing the item 'metadata' - not
> actually working on the item at all, but just tracking its state,
> progress, priority, etc. 
> 
> Planning as a task in itself is very important. I live by the 6Ps
> (Piss-poor planning prevents poor performance). However, that planning
> effort needs to be focused on moving things forward. Too often, I see
> people make the same error with org mode. They develop a complex all
> singing and dancing workflow for recording, tracking and managing their
> tasks. Before they realise it, they become a slave to the process and
> spend lots of time updating task state information, changing priorities
> and schedules, making notes which never get read again and tracking
> everything at a micro level. AS an extreme example, I saw someone once
> who had in their list of tasks "Get out of bed" - honestly, what is
> having that task really adding except noise and wasted time. Do you
> really need a task telling you to get out of bed? I've seen other
> examples where people find they are overwhelmed with the number of tasks
> in their task list, but when you look at what they have listed you see
> crazy micro level planning e.g.
> 
> 8:00 - Walk to bus stop
> 8:15 - Ride bus to Kent st. 
> 8:45 - Walk from Kent ST. buss stop to office
> 9:00 - Turn on office computer
> 9:00 - 11:30 - Do office work
> ...
> 
> Unless you have some sort of memory issue or learning condition,
> planning at this level is largely pointless. This may seem extreme, but
> I see people do very similar fine grained task logging in projects as
> well.
> 
> Planning, like effort estimating, is a skill and needs practice. All I
> am trying to convey is a warning against being overly ambitious or
> having overly high expectations regarding org mode and how/when it can
> be beneficial. While you can probably capture everything in your life in
> plain text org files, doing so may not actually help make your life
> easier. I believe you need to be judicious in what you map out as tasks
> and in what data you capture. Too much and there is a danger it all
> becomes white noise and you become a slave to the process. Our brains
> are amazing things which can handle a lot of complexity easily and at a
> fast rate - the interface is still faster and more powerful than org and
> we should use it to its maximum and apply org to augment it rather than
> replace it. Problem is, someone sees org mode for the first time and
> their blown away and suddenly can see how they can generate a
> comprehensive, concise and well mapped out process for everything they
> do. The pendulum swings to far the other way and suddenly, they now have
> an additional time consuming maintenance task which is now using time
> previously spent actually getting the job done.
> 
> The other danger is of course that we become too dependent on the
> technology. We can see this happening now with information like phone
> numbers. Even only 20 years ago, most people would memorise and be able
> to recall many phone numbers. Recent surveys indicate most people don't
> even know the phone number of their spouse! Consider that for a second -
> if you lost your mobile phone, how many of the phone numbers of people
> close to you can you recall? 
> 

-- 
 *Eduardo Suarez-Santana*
  Departamento de Computación Científica y Tecnológica
  División de Investigación y Desarrollo Tecnológico
 *Instituto Tecnológico de Canarias*, S.A. - Gobierno de Canarias

  Direc: Playa de Pozo Izquierdo, s/n - 35119 Santa Lucía, Las Palmas
  Telef: (+34) 928.727.583         fax: (+34) 928.727.597
  email: esuarez@itccanarias.org   url: http://www.itccanarias.org

AVISO LEGAL:
Este mensaje y los ficheros adjuntos si los hubiere, se dirigen
exclusivamente a su destinatario y puede contener información
privilegiada o confidencial. La transmisión errónea del presente mensaje
en ningún momento supone renuncia a su confidencialidad. Si no es vd. el
destinatario indicado, queda notificado de que la utilización,
divulgación y/o copia sin autorización está prohibida en virtud de la
legislación vigente. Si ha recibido este mensaje por error, le rogamos
que nos lo comunique inmediatamente por esta misma vía y proceda a su
destrucción.

LEGAL WARNING:
This message and the files attached if there were any, are intended
exclusively for its addressee and may contain information that is
CONFIDENTIAL and protected by professional privilege. A wrong
transmission to this message don’t suppose we relinquished to It’s
confidential. If you are not the intended recipient you are hereby
notified that any dissemination, copy or disclosure of this
communication is strictly prohibited by law. If this message has been
received in error, please immediately notify us via e-mail and delete it.

Re: Manual Ordering and Dynamic Priority

[Ihor Radchenko]

Eduardo Suarez <esuarez@itccanarias.org> writes:

> I have lots of tasks (todos) and I would like to create a long backlog based on
> my perceived priority.
>
> I was thinking to deal with them in the following way:
>
> - divide them in groups (categories or similar),
> - manually sort priority for every group,
> - mergesort groups, that is, start merging groups in pairs, and manually sort
>   for every step the union group until I have a large sorted backlog.
>
> For this to be practical, I would need an easy way to sort manually a group of
> tasks and get them assigned automatically a priority (or any other hack) so
> that priority ordering matches manual ordering.
>
> Any idea about how to get this done?

It sounds like you just want to have a list of active projects and
select one top-priority task at a time from each project. The projects
can be simply a headline with all the tasks listed as children.

I do not recommend assigning priority as a number. From my experience,
it is much easier to order the tasks manually inside a project. You will
need to re-order things as the project progresses anyway. Fiddling with
numbers will quickly become unmanageable.

I also do not recommend listing _all_ the tasks. It is much more
productive (again, from experience) to list the one top-priority task
for each project and nothing more until you finish the current task
batch. In GTD, this is usually done by assigning NEXT todo keyword to
the top-priority task, while all others are assigned as TODO. You may
also leverage org-edna package to flip TODO->NEXT once you complete the
current NEXT task.


-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

Re: Manual Ordering and Dynamic Priority

[Eduardo Suárez]

Thanks a lot for all your answers. I've been thinking about it and I've come up
with another solution.

My main problem is, given a file with lots of tasks, to create a "preorder" of
tasks to avoid the fear of missing out tasks when planning or selecting the top
priority tasks. To review the full list of tasks takes very long (my estimate
is O(n^2)). Reordering the current hierarchy is not an option because the file
has already a predefined (deep) structure.

So my solution is simple: create another top level heading in the file named
planning and create a hierarchy of links to tasks in it. Something like this:

* Planning
- [[#task4][My task 4 header]]
  - [[#task5][My task 5 header]]
  - [[#task1][My task 1 header]]
- [[#task2][My task 2 header]]
  - [[#task7][My task 7 header]]
    - [[#task6][My task 6 header]]
  - [[#task3][My task 3 header]]

So I can preorder using dependencies like org mode natural TODO dependencies,
and I get something like a gantt chart (without dates and planning). Moreover,
it looks simple to create a template to add a task to this list. I can even
document the relationship between tasks and priorities there.

Then, it is straightforward to select top priority tasks in this file.
Repeating the same for every agenda file, I can easily select a small list of
tasks to work with (e.g. assigning priorities or a TODO keyword).

Re: Manual Ordering and Dynamic Priority

[Ihor Radchenko]

Eduardo Suárez <esuarez@itccanarias.org> writes:

> My main problem is, given a file with lots of tasks, to create a "preorder" of
> tasks to avoid the fear of missing out tasks when planning or selecting the top
> priority tasks. To review the full list of tasks takes very long (my estimate
> is O(n^2)). Reordering the current hierarchy is not an option because the file
> has already a predefined (deep) structure.

If you really have so many tasks, something is not right. Normally, most
of the tasks are really optional and do not need to be reviewed often.
Only really important tasks should be reviewed regularly. Those can be
marked, say, as [#A] priority and reviewed in special agenda view/sparse
tree.

I also find it useful to apply spaced repetition for some lower-priority
tasks I really do not want to forget by accident.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92