Re: Adding Org Files to org-agenda-files

[Jean Louis <bugs@gnu.support>]

* Ihor Radchenko <yantar92@gmail.com> [2020-12-01 05:21]:
> Jean Louis <bugs@gnu.support> writes:
> 
> > We write tasks in their most logical chronological order and every
> > staff member is instructed to follow the order. One simply cannot
> > drive a car without putting petrol first, so that system is
> > followed. Some tasks on the ground can be done without chronological
> > order and our staff members are left to decide on that. When they
> > arrive to town and need to buy timber, maybe carpenter is discovered
> > right there while the task says that once they arrive to village that
> > they should look for carpenter. What is NEXT is mostly practically
> > decided while doing things at my side.
> 
> But what if the road to village is blocked by weather conditions? Should
> the staff members just stop doing the project and wait until the road
> becomes accessible? That sounds not very efficient. If all the tasks
> that one can start doing at current stage of the project are marked
> NEXT, instead of waiting for the blocked tasks, one can simply choose
> another NEXT task and get some progress on the project despite the first
> tasks cannot be done at this moment.

Just as you got a hunch, random incidents happen all the time on
ground. There is set of policies and staff members get trained to
apply them. For example our coordination policy is to pretty much
coordinate any reasonable action before, during and after
execution. If staff member is departing to village such will send a
message and we know what is the action of a staff member. If
supervisor is on computer such action can be entered in same central
file. Otherwise email list of staff member holds track of actions.

In that sense we help each other.

> Should the staff members just stop doing the project and wait until
> the road becomes accessible?

Actually I said contrary, we are problem solvers rather than
robots. Roads become accessible, we do not even speak of things that
have to be solved as it is self evident fact. While this case may
appear contradictory to you in real world people coordinate between
each other and help each other in real time.

Writing those smaller tasks on paper would be detrimental for a
project as more time would be spent to write the task or note the task
than to actually do it.

> If all the tasks that one can start doing at current stage of the
> project are marked NEXT, instead of waiting for the blocked tasks,
> one can simply choose another NEXT task and get some progress on the
> project despite the first tasks cannot be done at this moment.

After this discussion and review of how SMOS implemented NEXT and how
some people implement NEXT while doing their planning with Org mode,
I see that it will never be necessary on my side. Just never.

This is for reason that we use set of policies beforehand and train
people how to do projects. Number one is that person cannot start
doing any action without fully understanding all parts of the full
project. We expect person to be literate and capable at least in the
context of the project being executed. We push the purpose of the
project and reason, not the execution of single tasks. As purpose of
tasks are to achieve the purpose, person executing those tasks is
supposed to collaborate on the project and contribute to it. Executing
tasks is done by reason and not by robotic planning.

That should clearly answer why NEXT is completely redundant as in all
experience of years of planning, writing projects and assigning such
to people I have not even encountered a problem related to the subject
"NEXT" as used by people in Org planning:

- there is set of policies on how to train people for projects

- there is set of policies how to coordinate, communicate, report,
  including report on events

- plans have goals and purposes, projects fulfill one step of a plan,
  projects have its own purposes and tasks are there to complete a
  project

- any task becomes reasonably redundant if we have achieved the
  project target. Any project becomes redundant if plan's step or
  plan's purpose have been achieved. This is contradictory to
  robotic way of how Org have been programmed in relation to list
  items:

  - mark heading with TODO (let us say project purpose)
  - [ ] add TODO list items
  - only if all TODO list items are marked [X] the parent node can be
    marked as DONE

  That approach is contradictory to human logic of achieving things. I
  am not doing a single task for the single task's sake but for higher
  purpose and if higher purpose have been achieved, all those planned
  single tasks become reasonably redundant.

  I am using word "reasonably" as that involves human who decides
  about it and not robotic following of the tasks and executing them
  just because they may appear as not DONE.

- because we all get trained to use reason when handling tasks it is
  logical for the assignee to know what other task can be done within
  same section. It is completely redundant to mark tasks as NEXT and
  also not meaningful and confusing.

  Definition of "NEXT" is:

  1. (0) following, next -- (immediately following in time or order;
  "the following day"; "next in line"; "the next president"; "the next
  item on the list")

  And that implies to me and my personal logic that there cannot be
  more than 1 NEXT task to do. Marking multiple tasks as NEXT is
  contradictory to logic. I can understand it. But I cannot impose
  such logic onto other people reading it on paper as it contradicts
  to the its literate meaning.

  Meaning in the Org mode context is also pretty much idiosyncratic as
  each person may have slightly or substantially different
  consideration how to apply those action related tags or flags.

- When an assignee is reading the project all of the project is
  written in logical chronological order.

- Set of policies and training ensures that assignees are deciding
  themselves on what next task can be done, so we all do it this
  way. But marking or flagging tasks as "NEXT" is redundant as that
  would mean we would be dealing with robots and not with people. It
  seem to me like a low level programming without any intelligence.

When person really lacks some intelligence or is sent few first times
on a project, then more chunked chronological style is used:

1. [ ] Read this project and clarify all matters in headquarter before
   departing.

2. [ ] Make sure to eat and drink before the travel, what you can do
   in the city.

3. [ ] Call Magdalena and tell her you are coming to take 2 samples
   and to review the mining sites. Ask her if she can assist.

4. [ ] Find out from Magdalena how are you going to arrive from Kahama
   to the village, write information down.

5. [ ] Depart from headquarter.

6. [ ] On departure, send communication that you have just departed.

7. [ ] On any accident on the road, or event, during the travel, send
   communication of what is happening.

8. [ ] On arrival to Kahama, send communication that you have arrived.

9. [ ] Arrive to Kahama.

10. [ ] On meeting with Magdalena send communication that you met
    Magdalena and where you are located.

11. [ ] Travel to the processing site. Send communication that you
    arrived to processing site.

When person is trained satisfactorily then various reporting events
need not be written in the project as that is policy that is
implemented at all times and is named "Reporting on events".

- Projects have collaborative nature. Only if people know each other
  already well it may be advisable to write projects without much
  collaboration with other people or assignees. Org mode is by its
  nature more personal as there are no implementations for
  collaboration. In reality, when writing projects it is better to
  write such together with assignees as one is going on the ground to
  execute projects and other one is planning, but planning without both
  minds collaborating together is detrimental. Personally is good, but
  when two and more minds are involved in execution of projects then
  they shall at least be asked for opinions during the stage when
  project has to be understood before its execution. And their
  opinions may become part of the project.

- Tasks in a project cannot be distributed over different
  projects. While this is established as a policy I have never written
  it down because it was always established.

  On computer one can have many personal files distributed over many
  different Org files or other systems of keeping them. Then people
  use Agenda to find what is next to be executed. This may work well
  for one type of tasks, for example for group of tasks such as "Visit
  dentist", "Polish nails", "Go to cinema with Joe"; in general where
  each such task has purpose for itself but is not part of one whole.

  When a task is part of some higher order or higher purpose, then
  other tasks must be excluded as that becomes evasion or dodging. I
  can think how many Org mode users procrastinate what they really
  need with justifications of "doing something else" because "it is
  written in my agenda" as "my computer calculated it".

  There is a serious misconception that people face automatically when
  working with computer where things go into direction where they
  would not go if one would work with papers.

  When preparing a project on paper for example to negotiate a deal
  and assigning that project to person then person is "on the project"
  or "on the mission" and there is nothing else to think of.

  I am using this definition:

  3. (8) mission, charge, commission -- (a special assignment that is
  given to a person or group; "a confidential mission to London"; "his
  charge was deliver a message")

  Person assigned on a project is not supposed and maybe not allowed,
  and not directed to consider any list from any "agenda". Person does
  project steps and does not do "other tasks" or other project
  steps. It does not matter if maybe other task is scheduled for today
  or similar, everything is postponed and there is no need to think of
  it as person accepted to do the project and postpones automatically
  everything else. Attributes, tags, properties, all that becomes in
  reality meaningless.

That is reason.

What is not reason is to have unreasonable files of allegedly ordered
tasks which are in reality not ordered and proof for that is that
org-agenda exists in the first place. People do not keep their
projects and tasks in ordered manner and they need org-agenda.

That is why I almost never used org-agenda in last 5 years.

All my projects have been ordered logically and separated into
files. Files are named by projects. For example "Short Term
Prospecting Project" where person goes into nature and does some
activities of prospecting for minerals.

Higher order plan says among many other plan steps that person has to
be sent on "Short Term Prospecting Project". That is where project is
taken as such, printed and executed. It is executed on the
paper. Person checks out with signature and date that task has been
executed. Supervisor on distance could check it out in Org file. If
there is no supervisor, the project paper becomes its own
report. Report notes are written on the same project paper and full
report may be on the end. When finalized planner looks if project has
achieved its purpose or maybe need more work. Plan step is finished.

One person can be assigned to one project, other to other project.

Browsing through "agenda" for that would be detrimental and time waste
as project tasks are simply logically and chronologically ordered.

Both the assignee and assignor know, due to nature of project
planning, what is next. Neither of them is browsing to see what "could
be next" as both of them, the supervisor and people on ground, keep
same information in front of them and they can see what is checked out
as DONE and what else is may be reasonably and logically next to be
done.

I have no meaningless Org files. Files may be named as
How-to-place-posters-project.org and are usually one part of a bigger
plan and are usually exported from a section of a bigger plan (Org
file).

Several staff members have already executed
How-to-place-posters-project.org with success, have been earning money
and we have been all benefiting from that project.

Plans and projects on my side are programmed as in this definition:

The noun program has 8 senses (first 4 from tagged texts)

1. (106) plan, program, programme -- (a series of steps to be carried
out or goals to be accomplished; "they drew up a six-step plan"; "they
discussed plans for a new bond issue")

For that reason many various methods of how other people use Org mode
are redundant. NEXT is redundant. Searching through Agenda is
redundant for me.

What makes sense on my side is keeping track of following:

- which plan is currently ongoing?

  - what plan step are we on?

  - is the step too difficult that requires project planning?

  - if yes, what is the next project step to do?

  - supervise and help or execute the next project step

Practically if person works from day to day, there is nothing to be
asked as all above questions are known and project step is in front of
person's face either on paper or computer. There is no repetitive
rehearsing on what is next to be done and there is no searching for
accidentally distributed unrelated tasks in various other files not
ordered in this manner.

Because sometimes I am supervising people in several different
countries at once, then my personal set of tasks for a day is like
this:

- begin day by contacting all relevant people, greeting them, making
  sure they are all ready, coordinate and collaborate

- it is already known which people work on which project. We both know
  in the same time what is next. Marking anything as "next" at that
  point would be redundant. We may discuss maybe how to solve things.

- communication lines are kept and that step after step is executed
  while we are collaborating.

That way I can accomplish few different sets of projects in few
different countries on distance. Nobody is confused there. None of us
is searching through agenda as it is not relevant to search but do
what is written due to chronological and logical order of assignments.

While doing projects for business and humanitarian projects I have me
personal projects. In general I do not need to supervise people in
real time as they all report on the end of the day. That is where we
collaborate for tomorrow. After collaboration of some minutes I am
again free personally. 

org-agenda may be useful but it is on bottom of things, not on top of
things. Tasks in such planning do not belong anywhere, they are
distributed among files that are named any how where people do not
have any real method of sorting them. org-agenda will show then
anything, from personal tasks to business tasks, recreational, family
tasks or anything together and it does not make sense to me.

To be on top of things means to supervise things from higher
level. Think of countries and people involved. Because those people
are on different plans and projects that set of people makes a daily
list of tasks:

- Peter in Kenya, project on legalities, it is clear in the project
  what has to be done next. No agenda searching when it is already
  clear to both of us. It was written or collaborated and written in
  advance. Org file has been sent.

- Dean in Kenya, project of a service for client, it is clear what is
  next because it was written or because it was updated by
  collaboration from yesterday.

- Laurence in Tanzania, that is different project, same thing as
  above.

Higher planning comes first. Once planned, written, set on paper,
printed, I am not any more rehearsing those elementary tasks on paper
or on computer, they have been already written with practical
viewpoint, can be executed and are all useful. There is no need for me
to go through tasks again ever to see what is TODO, for example,
because it is already known. I am thinking of PROJECT ABC. I am not
thinking of TASK 42. No need to access on daily, hourly basis, or
rehearse tasks because it is usually already known and if maybe
forgotten or not memorized just open PROJECT ABC and see what it
is. I hope it is clear what means to work from top to bottom.

Working on Org file means working from bottom to top:

- make tasks, little here, little there, organize maybe by some
  groups, make this or that file, search through agenda because I have
  not ordered anything how it should be. Think of task first because
  it is scheduled for its own sake of being scheduled. Do the task
  because it is task and not part of one higher purpose. Mark flag,
  add properties, tag them to be able to search them.

The Org way of doing things is organizing procrastination with more
and more increasing complexities that are allegedly supposed to make
life easier.

Please do not stone me.

Humanity exists way longer than computers, and Org mode, planning
methods existed for thousands of years, and great projects have been
accomplished without any papers being invented at the time.

Human mind is capable of doing anything with or without
computer. There is plethora of people who live happy lives, accomplish
and finish all of their obligations and tasks of life, and do not use
neither computers neither papers.

Thus in my opinion there shall be less TODOs:

- Plan A, maybe related to business
- Plan B, maybe related to children
- Plan C, maybe related to house building, COMPLETED

Daily or weekly agenda should be the top level of a tree of things to
be done. It should be clear that in those hierachies all steps are
chronological and logical. What is arranged by time need not
necessarily be arranged by logic, so it is better using both terms
together.

Then the subtree of hierarchy need not be marked ever with TODO as it
should be clear that it is TODO. One can have nice markings, but one
should not allow something that is ACTION TO DO to be marked as NOT
ACTION TO DO, as that makes file editing error prone for project
planning. It is easy to do shift arrow left or right and to change
TODO to non-TODO and maybe even forget about it. If the task is sorted
under Project, such has to be done, if Project is under Plan it has to
be done.

One can mark tasks done as COMPLETED. Often this requires DATE, TIME
and SIGNATURE.

But marking project step with TODO when it is obviously sorted as
something to be done is not necessary.

Here is structure of a project, as part of bigger plan. Projects can
be structured any how on my side. When assigned to other people there
are sections of introduction:

1 Primary principle for reading ;; explains to people not to skip misunderstoods
2 Primary principle for communication ;; that we shall collaborate, etc.
3 Definitions of words ;; defines terms related to project
4 About company
5 Goal of the project ;; known objective, actions are done to achieve
                         the goal and it has clear quote
6 Purpose of the project ;; A purpose is a lesser goal applying to
                            specific activities or the sujects. It
			    often expresses future intentions
7 Requirements for this project ;; no moving to "TODO" without it!
8 How to do this project ;; explains how to conduct project, reason,
                            logic, collaboration is all here
9 How to report        
10 How to report on events
11 How to make pictures
12 Communication requirements [0/16] 
13 Personal introduction
14 Project steps            ;; this is where operational targets are defined
15 Awards

Project steps can be also related to various other objects such as
notes, videos, images, WWW hyperlinks, that is why now HyperScope is
to encompass that all.

A task type of heading marked as TODO in Org mode editing may be
related to other hyperdocuments such as images, text files, In Org
file we make hyperlinks to such rather external objects. If we start
placing them inside of Org file a single task becomes messy. We still
need such objects but not in a way to disturb execution of tasks. Such
objects are useful in preparations of projects, programs or plans for
each person to have references and to gain more understanding.

> > Concept is great: This task can be completed only when tasks 1, 4 and
> > 7 are completed. But practical life is different. When conducting
> > projects staff members may discover on ground that dependable task can
> > be completed without 1, 4 and 7 being completed as one could not
> > predict future random events. It may be also discovered that those 1, 4
> > and 7 are not true dependencies but some other tasks. This would imply
> > that planner must know very well future incidents which is rarely the
> > case as it would be so easy to predict future one would not be writing
> > tasks.
> 
> This can indeed be problem, especially if one tries to create too
> detailed dependencies. However, some very standard procedures might
> still benefit from this. For example, safety checklists might be the
> case when such task dependencies do make sense. Both the checklist and
> the dependency can be pre-defined as a capture template and then used in
> different projects serving as a reminder for necessary actions.

That is right. That is how I am planning everything. Not as
incidentally distributed tasks that I need to search with agenda but
everything is sorted into lists. From the list itself is clear that it
is action list. I am reusing those lists many times.

When you say "safety checklist" you are telling about senior element
such as purpose which is in this case safety and subordinate elements
which are elementary check list items.

There are those tasks which are scattered and Org file gathers them
all in Org agenda. So it tries to look through into the result of
organizing of procrastination. It may as well be that many Org users
do not have well defined senior purposes for those tasks and that many
tasks are not sorted in chronological and logical order. Then in
addition they are distributed in various files with file names not by
by its structure related to anyting in file systems that is also not
well ordered in itself and not related to anything in particular as
structured data.

Person can be collecting and gathering files on desktop in to somebody
else very messy way. But person doing it may have developed mind
mechanism to know where is what without using any particular system of
organization. All relations are kept in the mind. Putting structre
from the mind into computer brings organization that become capable of
collaboration.

Everything is related to everything in real world. We do relational
associations by using our mind. We just do not have adequate tools to
implement our mind association into our stored computer data.

But if we DO put attention that our computer data is logically related
to each other and well structured at that point there is no need for
organizing the mess as we then made one steap ahead for mess not to
appear in the first place.

That is why I say that Org editing is organizing of
procrastination. Some users will be doing it efficiently, but Org
agenda and tools developed for agenda show that demand is rather to
organize the creation of huge procrastination.

If things are well organized from ground up then agenda becomes
redundant.

Organized implies to me to know what is next to be done.

Unorganized person does not know what is next to be done. That is why
Org agenda is there. Because tasks are scattered, not organized.

> I personally use very simple edna dependencies - when there is a book
> series or a movie/documentary split into several series, I sometimes
> block the later series until I watch earlier:
> 
> https://github.com/yantar92/emacs-config/blob/master/config.org#task-dependencies
> 
> In any case, I suggested this package simply as an example how to make
> all subheadings become TODO as soon as one changes the parent to TODO
> state.

When looking into the real life beyond Org mode organizing, we may
find planners and administrators on much higher level unbelievable
organized that cannot compare to anything we are doing here. There are
cities on this planet well organized where administration even if
replaced with new people continues the plans that have been determined
decades ago. I do not believe they use Org, neither that they need
even to use any computers. Before the era of computers cities have
been planned and projects implemented and executed and nobody was
putting silly marks such as TODO. It should be clear from heading and
location of heading in overall structure that it is to be executed. If
it is not clear, something is generally wrong from ground up.

How did Chinese make the Great Wall without Org mode? How did they
bring the Statue of Liberty from France to New York and organized a
project?

Does military use TODO marks in their planning? I do not believe so,
it would look childish as their plans are already well chronologically
and logically structured that TODO, NEXT, etc. becomes redundant.

Each planning methodology requires something names goals or purposes
or objectives or targets and anything that has to be executed belong
to such goals. In military they will call them objectives. Myself I do
not approve of any wars neither military preparations, human animal is
crazy. But military planning methodology does not involve any random
searches over bunch of scattered tasks and data to find out what is
scheduled, etc. Army, marines, government officers in many countries
have methodology of planning that may be paper based or computer based
and outperforms any type of discussed Org established ways of
gathering the scattered.

Reference:
https://www.globalsecurity.org/military/library/policy/army/fm/19-4/Ch16.htm

,----
| Use the military planning and decision-making process.
`----

Org does not teach us about that. It tries to provide something from
elementary programming features, but not from top to bottom.

,----
| Develop long-range as well as short-range goals.
`----

This may be most important in any planning. Why are you doing what you
are doing? Is there maybe somethin else what you should be doing? It
is fine that one schedules visit to cinema with friends, but bills
will not be paid from tagging TODO on Cinema and spending wrong day
for activity that fits maybe in different week.

Thinking on long-range goal helps in determining short-range goals,
which help in determining which projects or tasks are to be executed.

Org mode has headings and hierarchy and established ways for people to
order their goals, projects, tasks, but it is not what people are
doing, because there is no form structure in Org mode to tell where
something is allowed to be ordered and where not.

That is why heading need its type. Type may be assigned by human
mind. But that is error prone. It is not suitable for collaborative
access as then one among the group could change PLAN type to be NOTE
without asking others or collaborating on that. Nobody would knew
it. Org agenda would not find that it is plan.

Heading with its type should not accept node that belongs somewhere
else. One cannot put family stuff and purchase of plastic swimming
pool for children under the submarine supervision plan run for
government.

They mention all times objectives, purposes goals:
https://en.wikipedia.org/wiki/List_of_military_strategies_and_concepts

Tasks have to be aligned to objectives, goals or purposes.

Org file better be aligned to its purpose or objective.

Headings shall belong to its senior heading.

In Org we do not have unlimited possibilities or it start to look very
funny. Org export to PDF is very fine and I mostly use it. But direct
LaTeX is more suitable for planning and better structured output. Org
mode tries to be glue for everything and has to degrade some
functionalities to make it easier and right for many users.

,----
| Identify your goals and objectives and the end point by which you will
| recognize their accomplishment.
| 
| Coordinate goals and actions internally and externally.
`----

That says a lot. Actions are those which we make as TODO. Are they
well coordinated with goals and objectives? If they are why are we
marking them with TODO as such? If they are coordinated and aligned
why we search for such with agenda?

,----
| Base your plans on objective planning factors.
`----

While each individual can do that in an Org file there is no structure
that teaches user how to do that. If Org planning would be done by
specific structure, at least templates to better organize. Organizing
actions means organizing it from top to bottom, from goals down to
single elementary steps that are executed with purpose to achive
goals. 

> > children nodes with the tag. It becomes very trivial when using
> > database with nodes having a parent:
> >
> > ,----
> > | UPDATE hlinks SET hlinks_tags = 'TODO' WHERE hlinks_parent = THIS ONE;
> > `----
> >
> > But rather a function would be used or type assigned. The above is
> > only example that shows how complex hard coded Elisp functions can be
> > replaced with 3-4 lines single function when database is a backend.
> 
> Why do you think that analogous Elisp function would be complex?
> 
> (defun yant/trigger-children (arg)
>   "Change all the children to TODO when parent is TODO."
>   (when (and (eq (plist-get arg :type) 'todo-state-change)
> 	     (not (boundp 'trigger-children-progress))
>              (string= (plist-get arg :to) "TODO"))
>     (let (trigger-children-progress)
>       (org-map-tree (lambda () (org-todo "TODO"))))))
> (add-hook 'org-trigger-hook #'yant/trigger-children)

Good for you, good for me. But not good as a product for people who
are not programmers. 

> > No wonder this guy has put Org mode in a sandwich on the logo of
> > SMOS. It eats the Org.
> >
> > SMOS - A Comprehensive Self-Management System
> > https://smos.cs-syd.eu/features
> 
> As for me, SMOS is too inflexible in comparison with org-mode. See
> https://old.reddit.com/r/orgmode/comments/ivlczu/smos_a_comprehensive_selfmanagement_tool/

It has defined machine parsable structure that may be used by any
programming language. It probably allows future or present
collaboration as it has server and client model. I have not verified
it. When it is so it allows for collaboration.

Org mode wants to become something that it is not. Gathering the
scattered. Being database while being plain text. Assigning built-in
relations while they do not exist. Organizing procrastination.

See CRM system like SugarCE Community Edition, it is free software:
https://s1.demo.opensourcecms.com/s/42

You will see how tasks can be created and they are structured by
foundational design:

- task may be related to person in structured database

- task may be related to account or company, bug, case, opportunity,
  etc. which all in turn could be related to contact, company, etc. 
  
- task can be assigned to person, there are scheduled dates and
  deadlines and there is also ability to expand structure with fields
  how one wants it.

In my opinion there is much to learn from there.

Task management with Org is limited to Emacs. It does not have some
centralized engine that could be used by other editors, but it should
have. At least it could expose its tasks for collaboration by using
export methods.

Even nvi editor could use external command to insert or update
specific part of a text. That would be more useful for
collaboration.

Maybe something similar to taskwarrior external program glued to Org
mode could handle better tasks and minimize hard coding and
reinventing the wheel.

With the central, outside engine for tasks then it would enable
collaboration. It could be editable from Org mode, but using external
engine.

Then users with any editor could access tasks and project
management. Web interface becomes possible. Collaboration opens.

There are many various free software task management programs from
where Org users could learn from:
https://github.com/awesome-selfhosted/awesome-selfhosted#task-managementto-do-lists

Majority of them have structured foundation.

Jean