Alternatives to clocking in/out for reporting time

Alternatives to clocking in/out for reporting time

[Russell Adams]

I make extensive use of timestamps for billing (timesheet)
purposes. I'm looking to automate this more, and I find the existing
clocking system inadequate. I'm hoping someone can point me in the
right direction.

Today I have log mode enabled so that each time I close a TODO item,
it records the date and time it was closed. At regular intervals while
working I add inactive timestamps to my notes. I've mapped that to a
single key, so it's quite fast. If I switch tasks, have an update,
made progress I want to note to myself, or leave and return I add an
inactive timestamp. I have well over 1000 inactive timestamps in my
current file.

Later I can open my agenda view on the working file, choose my
timespan (week or month), enable log mode to show when items were
closed, and then enable inactive timestamps to view all of the
timestamps. This itemizes all the events organized by time into a
timeline.

It's fairly straightforward from that timeline to count my hours based
on the record of where I spent my time. It is unfortunately a very
manual process.

I find Org's clocking to be too detailed, and that it doesn't play
well with dynamically organized hierarchies of notes. I frequently
create and close subtasks, or switch parts of the tree. Clocking each
one is too much overhead, and too granular. I don't need to provide
down to the minute reports of each item. It also doesn't appear to
allow rounding of values, so I still have to adjust the results.

What I envision is a way to count items in the agenda view to produce
a time report. Counting any inactive timestamp as 15 minutes, where if
a half hour or more is logged I round up to bill the hour. Closed TODO
items should count toward billing that whole hour. Clearly this should
be customized.

The point is that I'm not worried about accounting time by task,
instead I'm aggregating tasks into accounting by whole hours.

I'm looking at org-element, and it appears I'd have to do my own
agenda style scan of the whole tree to find items to classify by
hour. While I'm somewhat proficient at elisp, that sounds like a steep
wall to climb.

Is there an iterative way to review items in an agenda view so I can
do the math to produce a report?

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com
                                    https://www.adamsinfoserv.com/

Re: Alternatives to clocking in/out for reporting time

[Tim Cross]

Russell Adams <RLAdams@adamsinfoserv.com> writes:

> I make extensive use of timestamps for billing (timesheet)
> purposes. I'm looking to automate this more, and I find the existing
> clocking system inadequate. I'm hoping someone can point me in the
> right direction.
>
> Today I have log mode enabled so that each time I close a TODO item,
> it records the date and time it was closed. At regular intervals while
> working I add inactive timestamps to my notes. I've mapped that to a
> single key, so it's quite fast. If I switch tasks, have an update,
> made progress I want to note to myself, or leave and return I add an
> inactive timestamp. I have well over 1000 inactive timestamps in my
> current file.
>
> Later I can open my agenda view on the working file, choose my
> timespan (week or month), enable log mode to show when items were
> closed, and then enable inactive timestamps to view all of the
> timestamps. This itemizes all the events organized by time into a
> timeline.
>
> It's fairly straightforward from that timeline to count my hours based
> on the record of where I spent my time. It is unfortunately a very
> manual process.
>
> I find Org's clocking to be too detailed, and that it doesn't play
> well with dynamically organized hierarchies of notes. I frequently
> create and close subtasks, or switch parts of the tree. Clocking each
> one is too much overhead, and too granular. I don't need to provide
> down to the minute reports of each item. It also doesn't appear to
> allow rounding of values, so I still have to adjust the results.
>
> What I envision is a way to count items in the agenda view to produce
> a time report. Counting any inactive timestamp as 15 minutes, where if
> a half hour or more is logged I round up to bill the hour. Closed TODO
> items should count toward billing that whole hour. Clearly this should
> be customized.
>
> The point is that I'm not worried about accounting time by task,
> instead I'm aggregating tasks into accounting by whole hours.
>
> I'm looking at org-element, and it appears I'd have to do my own
> agenda style scan of the whole tree to find items to classify by
> hour. While I'm somewhat proficient at elisp, that sounds like a steep
> wall to climb.
>
> Is there an iterative way to review items in an agenda view so I can
> do the math to produce a report?
>

Russell,

I don't have an answer for your specific problem. However, I was
wondering if you looked at the options to configure the built-in time
clocking and report table?

The reason I asked is because I had basically the same/similar issue
when I was using Org to do my invoicing. I would have lots of sub tasks
which I would clock in/out and end up with far too many line items and
overly detailed reports. However, I was able to change this just using
tghe built in facilities of the clock table.

I've been meaning to finish cleaning up my org invoicing solution for
ages, but it is one of those projects which doesn't get high enough in
the todo list. I'm also not a great elisp coder (only barely
adequate). However, yu might find some of what I've done useful or might
point you in the right direction. See
https://github.com/theophilusx/org-invoicing

Unfortunately, I cannot guarantee that code currently works. It is code
based on what I use to use for my invoicing, which was pretty hacky, but
worked well for me. I started to pull it together to make it more a
usable module for anyone who needed invoicing support. To be honest, I
cannot remember how far I got before getting distracted by a different
shiny thing. What I can say is that the basic ideas do work and I
successfully used it for invoicing for some time (I'm now retired, which
is partly why I've not finished it off - I don't need to invoice clients
anymore). 

I do think that if you need to go down the inactive timestamp route,
your in for a lot of work. The main point about inactive timestamps is
that they are not involved in any reporting or processing. So, you will
need to write much of the code yourself if yo want to use that to
generate your invoices and calculate billing. You really need to rule
out the custom time table option first as that would be a lot less work
overall and probably easier to maintain. 

Re: Alternatives to clocking in/out for reporting time

[Russell Adams]

On Thu, Jun 30, 2022 at 07:26:20AM +1000, Tim Cross wrote:
> > The point is that I'm not worried about accounting time by task,
> > instead I'm aggregating tasks into accounting by whole hours.
> >
> > I'm looking at org-element, and it appears I'd have to do my own
> > agenda style scan of the whole tree to find items to classify by
> > hour. While I'm somewhat proficient at elisp, that sounds like a steep
> > wall to climb.
> >
> > Is there an iterative way to review items in an agenda view so I can
> > do the math to produce a report?
> >
>
> Russell,
>
> I don't have an answer for your specific problem. However, I was
> wondering if you looked at the options to configure the built-in time
> clocking and report table?

I've looked at it several times, and it just doesn't fit the bill.

Sorry.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com
                                    https://www.adamsinfoserv.com/

Re: Alternatives to clocking in/out for reporting time

[Samuel Wales]

a few things taht are probably all completely obvious or investigated
or irrelevant just in case.  just brainstorm.

do you have everything relevant in the same subtrees?  i.e. not
wanting granular, can you search upward for a dominating entry kind of
like git searching upward for the .git dir or so?  property drawer
could control what's a dominating entry.  you probably thoguht of this
or of having whatever categories as tags or categories in entries
though. in any case that would clock.  you could even have clocking
clock into that no matter wher eyou are via some timer in principle.
just a brainstorm.  you said dynamic som perhaps there is no
dominating entry for each category though.

org-time-stamp-rounding-minutes and org-clock-rounding-minutes .  i
presume you don't find them relevant.

reminder: inactive ts in the clocked notation is usualy treated
separately by org [i.e. not hte same type of ts] from bare ia.

perhaps some kind of text property in the following could categorize
entries and then org can sum them perhaps via some hook you'd
populate.  it might be possible in principle to re-create most of the
old agenda L timeline view, but with multiple files instead of a
single file and with a search filter, by using the search agenda view,
as there is some intermediate function, perhaps
org-agenda-before-sorting-filter-function.  idk if you neeed multiple
bare ia tses [or of various types of tses perhaps if you do decide you
need more than bare ia] per entry but doing so is probalby possible
merely by having that function add the duplicates.  the ts would be
put in the prefix format.  you'd have to have custom sorting by
whichever ts you are recording [in the total list] for each entry; idk
if that is straightforward.

above probably useless but just in case somethinghtere is relevant.

 6/29/22, Russell Adams <RLAdams@adamsinfoserv.com> wrote:
> I make extensive use of timestamps for billing (timesheet)
> purposes. I'm looking to automate this more, and I find the existing
> clocking system inadequate. I'm hoping someone can point me in the
> right direction.
>
> Today I have log mode enabled so that each time I close a TODO item,
> it records the date and time it was closed. At regular intervals while
> working I add inactive timestamps to my notes. I've mapped that to a
> single key, so it's quite fast. If I switch tasks, have an update,
> made progress I want to note to myself, or leave and return I add an
> inactive timestamp. I have well over 1000 inactive timestamps in my
> current file.
>
> Later I can open my agenda view on the working file, choose my
> timespan (week or month), enable log mode to show when items were
> closed, and then enable inactive timestamps to view all of the
> timestamps. This itemizes all the events organized by time into a
> timeline.
>
> It's fairly straightforward from that timeline to count my hours based
> on the record of where I spent my time. It is unfortunately a very
> manual process.
>
> I find Org's clocking to be too detailed, and that it doesn't play
> well with dynamically organized hierarchies of notes. I frequently
> create and close subtasks, or switch parts of the tree. Clocking each
> one is too much overhead, and too granular. I don't need to provide
> down to the minute reports of each item. It also doesn't appear to
> allow rounding of values, so I still have to adjust the results.
>
> What I envision is a way to count items in the agenda view to produce
> a time report. Counting any inactive timestamp as 15 minutes, where if
> a half hour or more is logged I round up to bill the hour. Closed TODO
> items should count toward billing that whole hour. Clearly this should
> be customized.
>
> The point is that I'm not worried about accounting time by task,
> instead I'm aggregating tasks into accounting by whole hours.
>
> I'm looking at org-element, and it appears I'd have to do my own
> agenda style scan of the whole tree to find items to classify by
> hour. While I'm somewhat proficient at elisp, that sounds like a steep
> wall to climb.
>
> Is there an iterative way to review items in an agenda view so I can
> do the math to produce a report?
>
> ------------------------------------------------------------------
> Russell Adams                            RLAdams@AdamsInfoServ.com
>                                     https://www.adamsinfoserv.com/
>
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: Alternatives to clocking in/out for reporting time

[Samuel Wales]

i said clocked notation, but i meant closed notation.  clocked
notation, i am not sure of.

On 6/29/22, Samuel Wales <samologist@gmail.com> wrote:
> a few things taht are probably all completely obvious or investigated
> or irrelevant just in case.  just brainstorm.
>
> do you have everything relevant in the same subtrees?  i.e. not
> wanting granular, can you search upward for a dominating entry kind of
> like git searching upward for the .git dir or so?  property drawer
> could control what's a dominating entry.  you probably thoguht of this
> or of having whatever categories as tags or categories in entries
> though. in any case that would clock.  you could even have clocking
> clock into that no matter wher eyou are via some timer in principle.
> just a brainstorm.  you said dynamic som perhaps there is no
> dominating entry for each category though.
>
> org-time-stamp-rounding-minutes and org-clock-rounding-minutes .  i
> presume you don't find them relevant.
>
> reminder: inactive ts in the clocked notation is usualy treated
> separately by org [i.e. not hte same type of ts] from bare ia.
>
> perhaps some kind of text property in the following could categorize
> entries and then org can sum them perhaps via some hook you'd
> populate.  it might be possible in principle to re-create most of the
> old agenda L timeline view, but with multiple files instead of a
> single file and with a search filter, by using the search agenda view,
> as there is some intermediate function, perhaps
> org-agenda-before-sorting-filter-function.  idk if you neeed multiple
> bare ia tses [or of various types of tses perhaps if you do decide you
> need more than bare ia] per entry but doing so is probalby possible
> merely by having that function add the duplicates.  the ts would be
> put in the prefix format.  you'd have to have custom sorting by
> whichever ts you are recording [in the total list] for each entry; idk
> if that is straightforward.
>
> above probably useless but just in case somethinghtere is relevant.
>
>  6/29/22, Russell Adams <RLAdams@adamsinfoserv.com> wrote:
>> I make extensive use of timestamps for billing (timesheet)
>> purposes. I'm looking to automate this more, and I find the existing
>> clocking system inadequate. I'm hoping someone can point me in the
>> right direction.
>>
>> Today I have log mode enabled so that each time I close a TODO item,
>> it records the date and time it was closed. At regular intervals while
>> working I add inactive timestamps to my notes. I've mapped that to a
>> single key, so it's quite fast. If I switch tasks, have an update,
>> made progress I want to note to myself, or leave and return I add an
>> inactive timestamp. I have well over 1000 inactive timestamps in my
>> current file.
>>
>> Later I can open my agenda view on the working file, choose my
>> timespan (week or month), enable log mode to show when items were
>> closed, and then enable inactive timestamps to view all of the
>> timestamps. This itemizes all the events organized by time into a
>> timeline.
>>
>> It's fairly straightforward from that timeline to count my hours based
>> on the record of where I spent my time. It is unfortunately a very
>> manual process.
>>
>> I find Org's clocking to be too detailed, and that it doesn't play
>> well with dynamically organized hierarchies of notes. I frequently
>> create and close subtasks, or switch parts of the tree. Clocking each
>> one is too much overhead, and too granular. I don't need to provide
>> down to the minute reports of each item. It also doesn't appear to
>> allow rounding of values, so I still have to adjust the results.
>>
>> What I envision is a way to count items in the agenda view to produce
>> a time report. Counting any inactive timestamp as 15 minutes, where if
>> a half hour or more is logged I round up to bill the hour. Closed TODO
>> items should count toward billing that whole hour. Clearly this should
>> be customized.
>>
>> The point is that I'm not worried about accounting time by task,
>> instead I'm aggregating tasks into accounting by whole hours.
>>
>> I'm looking at org-element, and it appears I'd have to do my own
>> agenda style scan of the whole tree to find items to classify by
>> hour. While I'm somewhat proficient at elisp, that sounds like a steep
>> wall to climb.
>>
>> Is there an iterative way to review items in an agenda view so I can
>> do the math to produce a report?
>>
>> ------------------------------------------------------------------
>> Russell Adams                            RLAdams@AdamsInfoServ.com
>>                                     https://www.adamsinfoserv.com/
>>
>>
>
>
> --
> The Kafka Pandemic
>
> A blog about science, health, human rights, and misopathy:
> https://thekafkapandemic.blogspot.com
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: Alternatives to clocking in/out for reporting time

[Russell Adams]

On Wed, Jun 29, 2022 at 07:11:10PM -0700, Samuel Wales wrote:
> a few things taht are probably all completely obvious or investigated
> or irrelevant just in case.  just brainstorm.

I appreciate that! That's really what I'm asking for is ideas. I don't
mind writing a bit of code, but I'm not sure where to start.

> do you have everything relevant in the same subtrees?  i.e. not
> wanting granular, can you search upward for a dominating entry kind of
> like git searching upward for the .git dir or so?  property drawer
> could control what's a dominating entry.  you probably thoguht of this
> or of having whatever categories as tags or categories in entries
> though. in any case that would clock.  you could even have clocking
> clock into that no matter wher eyou are via some timer in principle.
> just a brainstorm.  you said dynamic som perhaps there is no
> dominating entry for each category though.

The core issue is I want to report on total time as an aggregate, not
on time per task. Clock reporting gives me precise time accounting for
a specific task. I want to fill whole hours with whatever tasks
happened at those times chronologically.

> org-time-stamp-rounding-minutes and org-clock-rounding-minutes .  i
> presume you don't find them relevant.

I saw that. Interesting they don't appear to change the result, they
modify the input when you record the time by rounding.

> reminder: inactive ts in the clocked notation is usualy treated
> separately by org [i.e. not hte same type of ts] from bare ia.

I had considered perhaps converting inactive timestamps into clocking
records. Unfortunately I think the core issue remains in aggregating
by task, not by time.


------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com
                                    https://www.adamsinfoserv.com/

Re: Alternatives to clocking in/out for reporting time

[Samuel Wales]

make list of (ts . header) pairs [at least bare ia], rounding all tses
to hour, sort -u by ts, seq-group-by on ts to get (hour. headers)?  or
do i not get it?

On 6/30/22, Russell Adams <RLAdams@adamsinfoserv.com> wrote:
> On Wed, Jun 29, 2022 at 07:11:10PM -0700, Samuel Wales wrote:
> I appreciate that! That's really what I'm asking for is ideas. I don't
> mind writing a bit of code, but I'm not sure where to start.

usually opposite for me.  health reasons impede coding.

Re: Alternatives to clocking in/out for reporting time

[Ihor Radchenko]

Russell Adams <RLAdams@adamsinfoserv.com> writes:

> I find Org's clocking to be too detailed, and that it doesn't play
> well with dynamically organized hierarchies of notes. I frequently
> create and close subtasks, or switch parts of the tree. Clocking each
> one is too much overhead, and too granular. I don't need to provide
> down to the minute reports of each item. It also doesn't appear to
> allow rounding of values, so I still have to adjust the results.
>
> What I envision is a way to count items in the agenda view to produce
> a time report. Counting any inactive timestamp as 15 minutes, where if
> a half hour or more is logged I round up to bill the hour. Closed TODO
> items should count toward billing that whole hour. Clearly this should
> be customized.
>
> The point is that I'm not worried about accounting time by task,
> instead I'm aggregating tasks into accounting by whole hours.
>
> I'm looking at org-element, and it appears I'd have to do my own
> agenda style scan of the whole tree to find items to classify by
> hour. While I'm somewhat proficient at elisp, that sounds like a steep
> wall to climb.
>
> Is there an iterative way to review items in an agenda view so I can
> do the math to produce a report?

You may hook into timestamp insertion/todo-state change functions and
accumulate the "time" into headline properties.

I did something conceptually similar in
https://github.com/yantar92/emacs-config/blob/master/config.org#bonuspenalty-based-time-management

Hope it helps.

Best,
Ihor

Re: Alternatives to clocking in/out for reporting time

[Russell Adams]

On Fri, Jul 08, 2022 at 03:02:00PM +0800, Ihor Radchenko wrote:
> You may hook into timestamp insertion/todo-state change functions and
> accumulate the "time" into headline properties.

Hooking on insertion is a bit low level, and sounds like it's only
about maintaining state in the currently running Emacs.

I just want to get the agenda items in a programmatic way so I can
report on them.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com
                                    https://www.adamsinfoserv.com/

Re: Alternatives to clocking in/out for reporting time

[Ihor Radchenko]

Russell Adams <RLAdams@adamsinfoserv.com> writes:

> I just want to get the agenda items in a programmatic way so I can
> report on them.

Can you then formulate what exactly you want to achieve?
Do you want to consider only agenda items? All the timestamps in the
matching items or maybe just some?

Best,
Ihor

Re: Alternatives to clocking in/out for reporting time

[Russell Adams]

On Sat, Jul 09, 2022 at 12:00:53PM +0800, Ihor Radchenko wrote:
> Russell Adams <RLAdams@adamsinfoserv.com> writes:
>
> > I just want to get the agenda items in a programmatic way so I can
> > report on them.
>
> Can you then formulate what exactly you want to achieve?
> Do you want to consider only agenda items? All the timestamps in the
> matching items or maybe just some?

I typically use agenda for the month with logbook view and inactive
timestamps enabled.

I'd love to iterate over the list of all timestamps from that view.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com
                                    https://www.adamsinfoserv.com/

Re: Alternatives to clocking in/out for reporting time

[Ihor Radchenko]

Russell Adams <RLAdams@adamsinfoserv.com> writes:

>> Can you then formulate what exactly you want to achieve?
>> Do you want to consider only agenda items? All the timestamps in the
>> matching items or maybe just some?
>
> I typically use agenda for the month with logbook view and inactive
> timestamps enabled.
>
> I'd love to iterate over the list of all timestamps from that view.

You can. What you need to know about agenda buffers is that various
useful information is stored in text properties.

Every agenda line, which corresponds to a headline contains a non-nil
'org-hd-marker text property. In log mode, there will also be non-nil
'date and 'time-of-day text properties. You can examine the full list
yourself by running M-x describe-text-properties with point at an agenda
entry.

Hope it helps.

Best,
Ihor

Re: Alternatives to clocking in/out for reporting time

[Olaf Dietsche]

I also use org-mode clocking to record time spent.

But instead of modifying/configuring org-mode, I use a post-processing
Python script to collect and summarize clocked times.

    locale.setlocale(locale.LC_ALL, '')
    # ...
    def extract_times(line):
        # CLOCK: [YYYY-mm-dd WKD HH:MM]--[YYYY-mm-dd WKD HH:MM] => HH:MM
        m = re.search('CLOCK:\s*\[(.+? .+? .+?)\]--\[(.+? .+? .+?)\]\s*=>\s*(.+)', line)
        if m:
            ts = m.group(1)
            start = datetime.datetime.strptime(ts, '%Y-%m-%d %a %H:%M')
            ts = m.group(2)
            stop = datetime.datetime.strptime(ts, '%Y-%m-%d %a %H:%M')
            tm = m.group(3)
            spent = datetime.datetime.strptime(tm, '%H:%M')
            spent = datetime.timedelta(hours = spent.hour, minutes = spent.minute)
            logging.debug('time={}, spent={}'.format(line.strip(), spent))
            return (start, stop, spent)

Together with properties or tags or some other means, you can summarize
and round collected times as needed.
This can be done in any other (scripting) language, of course.

setlocale is needed here, because I have German weekday names.

Kind regards, Olaf

Russell Adams <RLAdams@adamsinfoserv.com> writes:

> I make extensive use of timestamps for billing (timesheet)
> purposes. I'm looking to automate this more, and I find the existing
> clocking system inadequate. I'm hoping someone can point me in the
> right direction.
>
> Today I have log mode enabled so that each time I close a TODO item,
> it records the date and time it was closed. At regular intervals while
> working I add inactive timestamps to my notes. I've mapped that to a
> single key, so it's quite fast. If I switch tasks, have an update,
> made progress I want to note to myself, or leave and return I add an
> inactive timestamp. I have well over 1000 inactive timestamps in my
> current file.
>
> Later I can open my agenda view on the working file, choose my
> timespan (week or month), enable log mode to show when items were
> closed, and then enable inactive timestamps to view all of the
> timestamps. This itemizes all the events organized by time into a
> timeline.
>
> It's fairly straightforward from that timeline to count my hours based
> on the record of where I spent my time. It is unfortunately a very
> manual process.
>
> I find Org's clocking to be too detailed, and that it doesn't play
> well with dynamically organized hierarchies of notes. I frequently
> create and close subtasks, or switch parts of the tree. Clocking each
> one is too much overhead, and too granular. I don't need to provide
> down to the minute reports of each item. It also doesn't appear to
> allow rounding of values, so I still have to adjust the results.
>
> What I envision is a way to count items in the agenda view to produce
> a time report. Counting any inactive timestamp as 15 minutes, where if
> a half hour or more is logged I round up to bill the hour. Closed TODO
> items should count toward billing that whole hour. Clearly this should
> be customized.
>
> The point is that I'm not worried about accounting time by task,
> instead I'm aggregating tasks into accounting by whole hours.
>
> I'm looking at org-element, and it appears I'd have to do my own
> agenda style scan of the whole tree to find items to classify by
> hour. While I'm somewhat proficient at elisp, that sounds like a steep
> wall to climb.
>
> Is there an iterative way to review items in an agenda view so I can
> do the math to produce a report?
>
> ------------------------------------------------------------------
> Russell Adams                            RLAdams@AdamsInfoServ.com
>                                     https://www.adamsinfoserv.com/

Re: Alternatives to clocking in/out for reporting time

[Samuel Wales]

On 7/10/22, Ihor Radchenko <yantar92@gmail.com> wrote:
> Every agenda line, which corresponds to a headline contains a non-nil
> 'org-hd-marker text property. In log mode, there will also be non-nil
> 'date and 'time-of-day text properties. You can examine the full list

idk if this is helpful but at one point long ago i /seemed/ to
occasionally need to do the following.

(or (get-text-property 0 'org-marker s)
      (get-text-property 0 'org-hd-marker s))))
        ;;do this to get to the org outline buffer from the
        ;;agenda buffer if you need other data that org-entry-get
        ;;cannot extract
        ;;
        ;;(b (and m (marker-buffer m))))
;;;        (if b
;;;            (with-current-buffer b
;;;              (goto-char m)